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I . INTRODUCTION 


Under NASA Grant NGR-47-004-116 , two major problems have been stud- 
ied. The first problem studied is the design of stable feedback control 
laws for sampled-data systems with variable rate sampling. These types 
of sampled-data systems arise naturally in digital flight control systems 
which use digital actuators. In these control systems, it is desirable 
to decrease the number of control computer output commands in order to 
save wear and tear of the associated equipment . Variable sampling also 
provides the designer with the capability of a more efficient utiliza- 
tion of the flight control computer than the standard fixed sampling-rate 
approach. Therefore, a variable sampling approach can also be of value 
to digital flight control systems using analog actuators. For instance, 
more time can be devoted to the Identification of aircraft parameters or 
to some other task by reducing the control calculations using variable 
sampling. 

The second major problem studied under NASA Grant NGR-A7-004-116 is 
the design of aircraft control systems which are optimally tolerant of 
sensor and actuator failures. The first problem to be resolved is the 
detection of the failed sensor or actuator. If the estimate of the state 
is used in the control law, then it is also desirable to have an estima- 
tor which will, give the optimal state estimate even under the failed con- 
ditions. Both the detection of sensor and actuator failures and the 
optimal state estimation with sensor and actuator failures are important 
control system problems which, if not resolved, can seriously (even 
fatally) degrade the control system performance of an aircraft. 
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II. OPTIMAL CONTROL OF SAMPLED-DAIA SYSTEMS WITH VARIABLE SAMPLING 

At each flight condition, the airplane dynamics can be modeled by a 
continuous, linear, time-variant, dynamic system [1]. Comparative simu- 
lations have indicated that a model-follower scheme in which the error 
between the model states and the plant states is penalized continuously 
in time was a more suitable approach in the design of control laws for 
sampled-data systems with variable sampling than designs based on a 
minimization of error at only the sampling instants. The aircraft dynam- 
ics are continuous and the gust inputs affecting the airplane are contin- 
uous random processes, but the measurements are made only at the sampling 
instants and the control is constrained to be constant between the sam- 
pling instants. Thus, the problem can be cast into the format of the 
stochastic sampled-data regulator problem of linear stochastic optimal 
control theory. The first problem to be resolved was to find out whether 
the separation theorem of linear optimal control continues to hold for 
the stochastic sampled-data regulator problem. This problem has been 
resolved and the results are reported in reference [2] along with a 
discrete-time stochastic problem which is equivalent to the stochastic 
sampled-data regulator problem. A summary of the results follows. 

The stochastic sampled-data regulator problem is to find the stoch- 
astic optimal control for the dynamical system represented by 

x(t) - Ax(t) + Bu(t) + w(t) tE[t^,tj^l (1) 

where x is the n-dimensional state vector, u is the r-dimensional control 
vector, and w is the white Gaussian plant noise vector of dimension n 
with Ew(t) = 0 and Ew(t)w' (s) = T^6j^(t-s) for some positive semidefinite 
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matrix E denotes the expectation operator and 6^ is the Dirac delta 

function. A and B are matrices of appropriate order. 

The plant noise is a continuous random process; however, the measure- 
ments are available only at the sampling instants: 


k = 0, 1, 2, . . . , N-1 (2) 

where y is the m-dimensional measurement vector, and v is a Gaussian 

'j 


y(tj^) = Cx(tj^) + v(tj^) 


sequence of uncorrelated zero-mean random vectors with Ev(tj^) v’ (t_, ) = 


'US,.. 

V kj 

The cost functional penalizes the state and the control continuously 
In time 


1 

J = Y u’(t)R^u(t)] dt 


(3) 


where is positive semidefinite and is positive definite. The stoch- 
astic sampled-data regulator problem is to find the control sequence 


N — 1 

{u (t, ) }, with the constraint 

' k = o 


u (t) = u(t^) 




k = 0, 1, 2, . . . , N-1 


(4) 


and also the additional constraint that u (tj^) will depend only on the 
past measurement sequence y(t^) , i = 1, 2, . . ., k which will minimize 
the cost functional (3). 

In reference [2], it is shown that this stochastic optimal control 
problem is equivalent to finding the optimal control for the discrete 
system 


^k+1 '^^‘^k+l’‘^k^^k ^^^k+1’ V^k '^k 


( 5 ) 


^k = <^k^k + ^k 


k = 0, 1, . . . , N-1 
where xj^ = x(tj^), u^ = u(tj^), = exp = 


4 


k+1 

/ ^(t, ^,s) dsB, and w, is a zero-mean Gaussian sequence of random 

k 

vectors with 




with the cost functional 

J - 1 + “k' W 


where 


^k+1 

\ ^ (t,tj^)Q^ $(t,tj^)dt 

k 

k 


( 6 ) 


(7) 


( 8 ) 


(9) 


k 

The stochastic optimal control is shown to be given by 

u.* = -[rT^m: + (r, + r!K._^Tr.)"^r:K.\, ?. ]x. 

X X X X X X+1 X^ X X+1 X X 


i = 0, 1, 2, . . N-1 


( 10 ) 


( 11 ) 


where is the solution to the Riccatx difference equation 

*^1 - h' f‘'i+r\+ih(h h' 

with = 0 and where = r(t = $(t .^^.t^-r . ' r'^' , 

Q. = Q.-M,R.^M, and x. is the conditional expectation of x. given the 
observation sequence (Vq, y^, • • •> y^}< 

Looking at equations (11) and (12), it is seen that the optimal con- 
trol law for the stochastic sampled-data regulator problem is the same as 
the deterministic sampled-data regulator problem [3] with Xj^ replaced by 
its estimate Therefore, the separation between estimation and con- 

trol continues to hold for the stochastic sampled-data regulator problem. 
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The only modification needed for stochastic control is the use of the 
derived equivalent discrete plant covariance (6) in the Kalman filter 
equations for the state estimates. 

As a byproduct of this investigation, new results were also obtained 
concerning the geometric relationship between the optimal solutions to 
the samp led-data and the continuous regulator problems. The findings are 
reported in A brief summary of findings is given below. 

Since the dynamical system (1) and the integral cost function (3) 
used are the same for both continuous and the sampled-data regulators, 
the optimal sampled-data control must be intuitively an approximation to 
the continuous one in some sense. The relationship between the two op- 
timal solutions, that is, the sense in which the sampled-data solution is 
an approximation to the continuous solution, has been obscured due to the 
separate formulations of these two problems in the control literature. 

The continuous problem has been solved by using the Pontryagin’s minimum 
principle, by using the Hamilton-Jacobi-Bellman partial differential 
equation for the optimal cost function, and by some other methods. The 
sampled-data problem has been solved by converting it into an unconstrain- 
ed discrete minimization problem through the integration of the cost 
functional and the system differential equations over each sampling in- 
terval and then applying dynamic programming or the discrete minimum 
principle. In our study, the two problems have been formulated in the 
same Hilbert space as minimum norm problems . In this geometric formu- 
lation, it is shown in reference [4] that the optimal sampled-data control 
is a "projection" of the optimal continuous control. Specifically, it is 
shown, that if [x* (t) ,u* (t) ] is the optimal continuous regulator solution 
and [x**(t) ,u**(t)] is the optimal sampled-data solution to the corres- 
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ponding deterministic regulator problems, then 


[Hx**(t), u**(t)] - [H4>(t,t^)x(t^) , 0] + 


( 13 ) 


Pj^[Hx*(t) - H$(t,t^)x(t^) , u*(t)3 
T 

where H is defined by H H = Q^, N is the sub-Hilbert space of all output- 
input pairs satisfying the sampled-data constraint with zero initial con- 
dition, and P denotes the projection operator with respect to the Hilbert 
space norm 


1 

|j(x,u)|| = 2 ^^ [x' (t)Q^x(t) + u' (t)R^u(t)]dt (14) 

o 

One of the implications of the result is that if an optimal sampled- 
data tracking problem is to be solved where the desired trajectory to be 
tracked is the optimal continuous regulator solution, then the solution 
obtained would be the same as the solution to the original sampled-data 
regulator problem. To phrase it precisely, the optimal sampled-data regu- 
lator solution is the sampled-data control that minimizes. 


1 

■J /j. {[x(t) - x*(t) ] *Q^[x(t) - x*(t)] + 

° (15) 

[u(t) - u*(t)]'R^[u(t) - u^-(t)]}dt 

The above equation clearly shows the sense of the approximation of 
the sampled-data control. Equation (14) shows that the optimal sampled- 
data solution is the projection of the optimal continuous solution onto 
the set of all solutions that satisfy the snmplcd-data constraint. 
Furthermore, the specific projections have been converted into recursive 
algorithms to compute the optimal sampled-data control. These algorithms 


are new; however, they are not necessarily less complicated than the known 
Riccati equations in the literature. 
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Stochastic Modeling Approach to Variable Sampling 

During the course of the project, a number of sampling interval 
adaptation control laws have been developed based on minimization of 
local objective functions. Although these algorithms proved to be 
successful in various simulations, they suffered from being only locally 
optimal. A control law that would take into account the changing of the 
sampling intervals was needed. This was accomplished by modeling the 
sampling interval sequence as a finite-state Markov chain with known 
transition probabilities. The finite-state assumption, that is, the 
constraint that the sampling interval can assume values only from a 
finite number of sampling intervals, was necessary to avoid an infinite 
set of equations. Specifically, the system equations are given by 


x(t) = Ax(t) + Bu(t) 


(16) 


where x, u, A, B are as in equation (1). The sampling interval sequence 
{Tj^, k = 0, 1, 2, . . . } is a finite-state Markov chain that assumes 
values {S^, S^, . . ., S^} with the transition probability 


P(T, = S. T, = S.) = P. . 
. k+1 j ' k X _ xj 

and the initial probability distribution 


(17) 


P(T„ = S.) - P , i = 1, 2, 

U X X 


n 


Then, the discrete-time stochastic system is given by 


(18) 




k = 0, 1, , . ., N-1 


( 19 ) 
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where 4>, P are as in equation (5). The observations are modeled as 

That is, at each sampling instant tj^, the state vector at t^^ is 
known and the sampling interval Tj^ to be applied at the instant tj^ is 
known. Note that the sampling intervals to be used at and further 

on are uncertain at time tj^. Only a statistical knowledge of their un- 
certainty is known through the transition probabilities. Measurement 
noise and plant noise are not included in the model to keep the equations 
simple. The problem is to find the stochastic optimal control 
{Uj^, k = 0, 1, . . ., N-1} that depends only on the past and present 
measurements i = 0» 2, . . . , k] that minimizes 


N-1 


k=0 


( 21 ) 


The stochastic optimal control has been obtained by dynamic pro- 
gramming and is given by 

where F(T.) = F, . for T, - S., given by the recursive equations 

iC 1^ 9 X cC X 


F, . = [R 
k,x 


J J • 


(23) 


n 




)(«.-r.F, .) + F’, .RF, . (24) 


r . ^ i *. 4 • 

X X k,x 


with ^ = H; i = 1, 2 , . . . , n; k = 0, 1 , . 

Ti = r(s^, 0 ). 

That is, at each sampling instant tj^ the control is given by a 
linear feedback law and the feedback gain at the instant t^^ is ^ where 
the "i" corresponds to the sampling interval S . at the instant t, . The 


k,i k,i 


, N-1; and = $(S^,0) , 


recursive equations are similar to the standard Riccatx equations of 
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linear optimal control. As it can be seen from equation (24) , n coupled 
Riccati equations are to be solved recursively. The coupling comes from 
the terms > corresponding to the transition probabilities for the 

sampling interval sequence. In the case when n = 1 and = 1, we get 
the standard Riccati equation of optimal control. 

Steady-state gains can be obtained as k ->• then lim. k *+ -“> 

F, , — F . , i = 1 , 2 , . . . , n . 

tv y JL JL 

The stochastic optimal control for the infinite time problem becomes 

u, = -F.x, for all k Such that T, = S, . 
k 1 k k X 

The infinite time optimization solution is very easy to implement. 
Steady-state gains, {F^, i = 1, 2, . . ., n}, where n is the total number 
of possible sampling intervals, are calculated off-line and stored. At 

each sampling instant t, the feedback gain F. is used where "i" is de- 

iC X 

termined by the sampling interval S^, at time t^. The algorithms for the 
stochastic optimal control, equations (23) and (24), are not yet avail- 
able in the literature. 

The difficulty of applying these results can be the selection of a 
sampling interval adaptation law, since the problem formulation does not 
address that question. However, for any sampling interval adaptation law 
whose sampling interval sequence can be modeled by a Markov chain with 
appropriate transition probabilities, the stochastic optimal control de- 
rived can be used. Moreover, several examples have been simulated on the 
digital computer where the sampling Interval sequence was chosen arbi- 
trarily and the results have been very successful. A computer program 
that calculates the stochastic optimal control gains has been written. 
Given next is an example showing how the results can be applied to a 
specif ic problem. 



xo 


Example 

Let the system matrices be given by 



— 



“ - 


0 

6.28 


0 

A = 

-6.28 

-3.14 

B = 

1 





— 


( 25 ) 


The control weighting matrices have been chosen to be 

1 0 

0 1 


Q = 


R = [1] 


(26) 


Sampling intervals are chosen as 


«1 = -1 


^2 = 


(27) 


The transition probabilities have been chosen to be the simplest case 

= .5 i,j = 1,2 (28) 

That is, if the sampling interval at is S^^,. the chance of the next 
interval being the same and the chance of the sampling interval jumping to 
S 2 are equally likely. 

The stochastic optimal control gains have been solved by using equa- 
tions (23) and (24) to obtain the steady-state values 


= [-.0181 .4455] 

=[.01246 .0612] 

The stochastic optimal closed-loop system has been simulated by 
using arbitrary sampling interval sequences, Figure 1 shows one of these 
runs. For this fixed sampling interval sequence, the deterministic op- 
timal control feedback gains have been found also, and the closed-loop 
simulation of this system is in Figure 2. An overlay of the two figures 
would show that the two trajectories are virtually the same. For a 
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fixed sampling interval sequence and the cost function of (21), expecta- 
tion can be dropped when the sampling Interval sequence is fixed because 
the system becomes deterministic in this case. Since this resulting 
deterministic time-varying linear quadratic regulator solution is optimal, 
the closeness of the stochastic and the deterministic trajectories are 
quite noteworthy. The remarkable aspect of the stochastic optimal control 
is that it not only matches the ultimate optimal deterministic performance 
for this fixed sampling interval sequence, but it also matches the per- 
formance of several other sampling interval sequences in the simulations. 
Of course, the deterministic optimal control is not realizable for thva 
problem we are considering since it requires the absolute knowledge of 
the whole sampling interval sequence. 

Stability of Sampled-Data Systems with Variable Sampling 


The deterministic stability conditions of the stochastic model of 
the previous section has been investigated jointly with Dt. D. P. Stanford 
of the College William and Mary. The results have been reported in [5]. 
From the formulation in the previous section, it is seen that "n" closed- 
loop discrete system matrices will be obtained 




where ”n" is the total number of possible sampling intervals and the 
are the stochastic optimal steady-state gains obtained through equations 
(23) and (24). The question of whether any initial condition can be 
brought to zero by a repeated application of the matrices in (29) in some 
order has been investigated in the stability analysis [5]. If a sequence 
of matrices can be found whose terms are selected from the set of matrices 


in (23) for each initial condition such that the state goes to zero in the 
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limit when this sequence of matrices are applied, then the set of matrices 
will be called convergent. In [5], it is shown that ’’precontractiveness" 
is a necessary and sufficient condition for a set of matrices to be con- 
vergent. It is also shown that "contractiveness" is a sufficient condi- 
tion for a set of matrices to be convergent. The stochastic optimal 
feedback gains have been found for a number of examples. It has been 
found that the closed-loop system matrices resulting from the stochastic 
optimal control turn out to be contractive in each case. However, an 
analytical justification has not been found yet. 

Ill . FAILURE ACCOMMODATION IN CONTROL SYSTEMS 

The second major problem which has been investigated in NASA Grant 
NGR-47-004-116 is the design of aircraft control systems which are op- 
timally tolerant of sensor and actuator failures. A design method has 
been developed, and the results have been reported in Ref . [6]. The 
method developed is based on Bayesian decision theory. 

Each sensor and actuator failure mode (including the normal opera- 
tion mode) is formulated as one hypothesis. Using M-ary hypothesis 
testing, the corresponding likelihood ratios are computed for each 
hypothesis. The computations of likelihood ratios require M different 
Kalman filters corresponding to M different failure modes of the system 
where M is the total number of different hypotheses. By comparing the 
likelihood ratios, the most likely failure mode of the system is selected 
in the Bayesian sense. 

The unique feature of this method is the flexibility of modeling the 
sensor failures as noise with unknown mean and variance. The mean of 
each sensor in a failed mode is computed on line by employing maximum 
likelihood estimation. This estimate for the mean is used in the calcu- 
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lation of the likelihood ratios which make them generalized likelihood 
ratios. The advantage of the method is that it is not required for the 
designer to know how the sensor will fail . This point has been demon- 
strated in the simulation [7] by the superior performance of the system 
in the detection of both increased noise type and hard-over type failures. 
'Although the variances of the sensors in failed modes have been fixed 
in Ref. [6] , the same approach for the estimation of the mean can be used 
for the variance as well. The maximum likelihood estimation of the para- 
meters requires the storage of a moving window of innovations of each 
Kalman filter. 

The applicability of the fault-tolerant system design has been demon- 
strated by using a real-time hybrid simulation for a space shuttle orbiter 
developed jointly with Dr. R. C. Montgomery of NASA/Langley Research Center 
The failures were identified in two or three sampling periods. The simu- 
lations indicate that the use of steady-state Kalman filters were adequate. 

Parameter Adaptive Estimation 

From the preceding failure detection problem in aircraft control 
systems research, the additional benefit of obtaining the optimal state 
estimate under failed conditions resulted. This problem is exceedingly 
important if state variable feedback is used and if a filter is used to 
get the state estimates. This estimation problem has been resolved in a 
Ph.D, thesis [7] and extensions of the work have been reported in [8]. 

This problem is kno\m as parameter adaptive estimation in tlie literature. 
The following is a summary of the work reported in [7] and [8]. Consider 
the observation model 

y(t) = z(0,t) + v(t) (30) 
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where G is a random variable and, for each fixed value of 0, z is a ran- 
dom process and v is a white Gaussian random process. The problem is to 
find the minimum mean-square estimate of the signal z. Detection problems 
can be easily modeled by this formulation with a suitable choice of the 
random variable 0 [7]. The general case when 0 has an arbitrary distri- 
'bution has been worked in [8], When the parameter 0 has a discrete 
distribution 

P(0 = 0^) = P^, i = 1, 2, . . (31) 

it is shown in [8] and [9] that the minimum mean-square estimate of the 
signal z is given by 

00 

z = E ir (t)z. (t) (32) 

i=l 

where z^(t) is the estimate of the signal z(0^,t) given the observation 
y(s) = z(0^,s) + v(s), 0 ^ s ^ t and where is given by 

Tr^(t) = P(0 = 0^ I y(s), 6 <_ s ;< t) (33) 

That is, ^z^(t)}^_j^ are the parameter conditioned estimates and 

OO . ' 

{iTj^(t) are the posterior probabilities of the parameter 0. 

The posterior probabilities satisfy the stochastic differential 
equations 

CO CO 

dTT (t) = TT.(t)[z (t) - E ir (t)z.(t)]R ^(t)[dy(t) - E Tr.(t)z (t)] (34) 

j=l ^ ^ ^ ^ 

with the initial conditions 

ir^(0) = P^, i = 1, 2, ... . (35) 

The parameter conditioned estimator has two parts: Anon-adaptive 

part in which the parameter conditioned estimates are found and an 
adaptive part in which the posterior probabilities are found. The form 
of the solution given by (32) and (34) implies that if recursive equations 
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are known for the parameter conditioned estimates, then a completely re- 
cursive solution is found to the problem. The recursive form of the so- 
lution is, of course, very advantageous in terms of implementation of the 
filter. The following example demonstrates how these results can be 
applied to a nonlinear filtering problem, 

. Example ; Consider the random telegraph signal, Z(t), with values ± 1 and 
transition density a where a is a random variable with prior distribution 
P(a = a^) = i " 1, 2, . . . , M. The observation model is given by 

y(t) = Z(t) + v(t) 

where v is a unit-variance Gaussian process, from (32) and (34), it is 

seen that the minimum mean-square estimate of the signal is given by 

. M . 

Z(t) = E n.(t)Z (t) 
i=l ^ 

where the parameter conditioned estimates Z^(t) are kno\'m to be given by 

IS] 

dZ^(t) = 2a^Z^(t) + [1 - zj(t)] [dy(t) - Z^(t)dt] 

which, in turn, derives the stochastic differential equation (34) 

M ^ M 

diT (t) = IT (t) [Z (t) - E IT. (t)Z (t) 3 [dy(t) - E it (t)Z (t)dtj 

j=l ^ ^ j=l ^ — 

with •iT^(O) = P^, i = 1, 2, . . M. 
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Directions for Further Research 

In the work on optimal sampled-data regulators, a promising and in- 
teresting problem is to find early computable algorithms to perform the 
projection in equation (13). In this way, optimal sampled-data regulator 
gains can be obtained as a function of the optimal continuous regulator 
gains. So far the equations obtained are not any simpler than the known 
equations available in the literature. However, this study can lead to 
an understanding of how the sampled-data optimal gains vary with the 
sampling Interval even though the algorithms may turn out to be complicated 

In the work on the stochastic modeling approach to variable sampling, 
more analytical and experimental study is needed. Analytical conditions 
that guarantee the convergence of gains for the steady-state gains 
[equations (23) and (24)] would be very useful. Simulation of a wide 
range of applications to different problems is needed to see how far the 
stochastic optimal law can be "stretched." So far, the simulations in- 
dicate that the stochastic optimal control law gives virtually the 
ultimate deterministic performance. 

In the work on parameter adaptive estimation, more work is necessary 
on the following questions. What happens when the parameter does not 
have the prescribed distribution? In this case, for instance, does the 
set of posterior probabilities converge to a value that is nearest to the 
actual value? 

Also, additional effort is necessary to understand the stability 
characteristics of the stochastic differential equations for the 
posterior probabilities (34) . 
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A separation theorem for the stochastic sampled-data 
LQG problem! 

XKSI.M IIALVO! AU>KR K. r.ACJLA V.\X§ 

'I’lii.s |iii|iiT c’linsiiliTH (111’ ciiiitritl mC ii cniii imimis liiii-ar pliml (list iirl»‘<l liy vhiU' jiliml 
iKijsc uIk’Ii till' Odtilnil is l•Ol)sl rdiiic'l to Ijf a piecewise cuii.sliiiil rniiclion of lime; 
I.e. u stdclmslie snnif)lc<l-(liitii sy.slein. 'I'he ensf fimc'Hiin is llie inii'griil of (|imtlnitie 
error (ernis in llii' ,stu|e luiil conirdl. Unis pcimlizinii I'rror.s nl every iiisimil of lime 
while (lie plmit noi.se (fistiirhs llie .system eont imidiisly. 'I’he prohlem i.s solved hy 
redocinj' the foiistniioed contiiltidii.s |irdhli'm to mi mieonsl rnineil discrete one. It 
is shown that the .sepnrididit principle for e.slinmUdn and control .still hoki.s for this 
prohlein wlien the plaid dislurhanee aial rneasurcinent nois<‘ arc iiatissian. 


J. Introduction 

VcTi'ioiis Ktiidirs of the (Ipterminintic diKcroto-limc re'guliitor problem have' 
been mndc Kinco the orijrinul work of Kaiimm and Koepeko (lUbS). A tutorial 
review of thcdc basic resiilt.s can be found in flic jjapor by Dorato and Jx'vi.s 
(1971), The deterministic .Siimi)lcd-(lata regulator prohlem has al.so been 
studied by Id'vi.s ef aL (1971). 'I’ho stocliastic counterparts of the eontinuou.s 
and di.serete linear tpiadratic {laussian [najillems restilt in the sejiunitimi 
jirincijilc for estitiuitinn and control (Joseph and 'rnu HKi], fiunekel aiul 
ITankiin I'.HiJ, I’nltcr fOd-t). Wnriou.s e.vten.siuns of the .separation (lieorem 
(0 include ti itirLrer cla.ss of continuous and discrete eosi funetions btive idso been 
made (c.pu iStreibel l!Hi.7, Wonlmm !9(iN, Halyu iincl l-'ouikes 197-1, ami (be 
references tlierein). 

In (his pajier the control of a stoeha.stie linear siunitled-flaia .system (i.e. 
tlie .sloelia.stie (‘ounteipart of (be work by Lt‘\i.s it nL (1971)) is eunslileml. 
A eontiniums linear plant is disturbed by a eontinunus (('nu.s.sian while plmit 
noise, wliilc the euiilrol is con.strainetl to In- cnn'-lant in hetueeii .samjdiiid 
instanfs, say {//_}. hut can eliiuifre at every sampling instiuil. .Metisuieimmls, 
which can he exprcs.sed as linear combimitioiis of the stale variables eonuptod 
by (!au.ssian white iiui.se, are made at (he .sampling!: iiistant.s. ,Such .sy.stems 
are often eneountered. iiartieularly if the eoiiftoi law is to be imi)!emented on 
a diifital computer. Oii the other hand, the peiformanee rd the .sv.stem depends 
on the deviations of the .stale vector from a steady-.stale eunditimi (rejire.sented 
a.s (he 7.ero vector) at e\'ery instant of time, not only at the .samiiliny instants, 
lleiico, a co.s( fundion which [Hmalize.s dt*vialion.s in (lie .state continuously is 


Heceivcfi 2.S Ajiril li)?;"). 

t 'J’lii.s Moik Was 'iU|ipnrlcd Iiy NA.S.A banalcv Ke-scardi Center under (Aiutrad 
X.-VSl- inL'Oatal {.'rani 17 not -I in. 

:|; Di’partiiieiit of Kleetiieal iMipneerinp, Lhu\’ers.ity nf \’iiyiiiia. Chariot tcsville, 
\'irgiiiia. 

§ Uepartineiit of ICli-etrieal Mugim eriug, ^'il•gillia i’ulytechuie lii.* .Hiiie, Hlavks- 
hiii-g, Vii^giiiia. 
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more I'lpproiii'iiilt' ;is a pcrformanc'c friU'i'imit l!uui a disfrclo cost fiinctinn. 
Tlio ]irol)lc‘in is snlvofl Ip mhicin^ it- to a diKm-tt' .slncliaslia ri’>i;uliitor where* the 
ptatcK are measured with no C’lTor. 


2. Statement of tlic stochastic sampiccl-datn prohicm 

Consifler the foliowincc continuous sloeliastic dynamical system represented 

by ‘ ■ ‘ ‘ 

m^Amn+B{ih>{i)+fmnii), munit] (n 

where ,r is the /(-dimensional state vector, ii is the /•-dimensional control vector, 
aiul //• is the white (laii.ssian |)lant lu/ise y(*clnr of dimension // ‘witli /wc(/) = 0 
auul A’//'(t)?e'(.'?) = — .s) for soim* posilivo semidefinite matrix F, A, B 
and 1) are time-varying matrices of compalil/le order. Kfjualion (1) i.s inter- 
preted as tlm following .stochastic integral ('<(uatiim (Wong Itl7l) ; 

I / / 

.r(/)=,r(/„) -f J A{.’i).r{s)d''iA- J A (.s )//(,?) r/s -f J dWi-t) (2) 

fn 

where. H'(0 is a Wiener process with A’ir(/) = 0 ami 

ruin (/, «) 

AXir(/)-M'(g)(H'(*^)-ir{g)'== i mdr 

a 

'I’he plant noise can he considered a.s the formal time derivative of the Wiener 
])rocc.ss If. 

In the sinchastic sainplcil-data |■cgu!ator (he ]ilniil noise is a contiiuinii! 
r.'indom proee.ss, whereas the mea.surement noi.se is a disereti* ramlnin profC« 
Measiireinents of some of the linear eomliinations of stales with additive not« 
areavailableatthesamplinginstants 

//(g=cV(g-t-r(/j, A-u. 1 V, (3i 

where.// i.s (he ///-dimensional ineasurcinenl veelor, ('■(//.)] tin* mea.sure)il'’i;‘. 
noise veetor which is a CJau.ssian .se(|uence of uncnrreliited -/.ero mean raiKh"'. 
veetors with A'rf/, )/*'{/;)--- for some pn.silive definite matri.x d,, ai/i 
A;ilX/)/'X/d= = = « b/r all t in |/n, /fjp-ind i = U. 1. ...,y. 

The controls are constrained by ! 

?/(/) = A=::0, t, .V- t (t 

In order for a .solution to exist to ecpi. (2), the followine fiip) lier a.ssiinij)tK>:.i 
will l)(' made : tht> element.s of d(f) aio luuiiuled and measuial/le real fuaclii'.“> 

ii 

of time, /f(/) is integrahle, and C bnile. 

i. 

+ If tlu* samplin.u rale can lie clui'^eu hi.gh enoiuih a discreie ciet fam-liim am) *» 
!ule(|iiate t hm\ I'ver. in man.v ca.-'es this increase^ (he eust nf 1 he eempnli'r Iw phriv 
slrill.geiU reijuil’eiiients nil its .'-peed nf tiperal inn. 't’lii.s Irade.nff inal-,cs llir ilr.<.r> 
of cnntrol laws, which do not degrade at Inw saiu|iling I'iitcs, iin|uirt:mt. 
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1’hc follouiii>i cost fuiu’tiniial is used lo achieve the desired system perfor- 
niiincc ; 

d 


d^UC 


J [y{l)Q,[l).v[l)+ it'aMiADuil)} dl -V .r'(/f)//.r{/,) 






where Q,. is positive semidefiiiite and /i, is positive definite on l/„, /(]. 

Now tlie stoehastif nptiinal control prohlctn can lie stated as follows ; 
given the linear stochastic dynamical system (I), and a i)artitinn 
uf the interval ') control sc(piem’c (»*(fy,), A' - n, I,..., >V-1) with 

the constraint (4) and also the additional constraint that «*(/,,.) will clejicnd 
only on the past incasnremcnt scfjiicncc (//(/,), i = 0, l,..,,A‘-l) M’hich will 
minimize the cost functional (o). 

3. Equivalent stocliastic discrete-time problem 

It will 1)(' shown that the constrained continuous stochastic optimization 
lirohlem can he transformed into an unconstrained discrete' stochastic optimi- 
zation jiroblem hy integrating; the system diffeivntial cipiations and the cost 
functional over each sampling interval. 'I’hc problem will be embedded into 
the known format of the standard discrete linear (jiiadratie (Jaussian regulator 
problem. Tlius, it will be proved that the separation bi'tween estimation and 
control is .still valid for this eon.strained eontinuovi.s st<jchasti(i optimization 
problem. 

Under the assumptions made in ^ 2, the .solution to eipn (1 ), for any bounded 
control, is given by 

•)'(0 -<l>{/, /„).!’(/„)+ I <\Ht,s)li{s]tt{s)ds-i- f d3(h y)/J[.>‘) dll'(.s’) (li) 

0 I. 

where <hi/, ■'<') is the state-transilion matrix of -1(/) defined as tlm solution of 
luatri.x differential e(iuation 

-^d>(/,6-) = ^l{0d>U,6), {l>s) 

III 

with ‘h(s, A') - 

Using (ti), the state for 1,/$/ <t/, n CD'i be computed from the state at time /^. : 
where .T;.“.r(/j.), = »(/).), and 

I <Hl,s)mds (7 6) 

a 

f 'i)(/,.s)/)(x)./ir(s) for 0<‘) 

a 

The term given by eipi. (7 i*) show.s that the plant noise corrupts the .stale 
in a continuous fashion, but the resvdting discrete-time system will he given b^' 

4V, i = <!>(/* (« «) 
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wiioro ~ , |, tj) i a moan whilo (laiissinn ficcjiionoo of ramlom voofors 

wUIi (Jazwinski 11)70; 


i> 




(s^) 


Tito assiimiil ions madt' ahotil llio iii(lo|)onilonoo (»f |ilanl mtiso and HI/,), Htn) 
in 5; - "‘ill •‘^till bo valiil fui tlio lU'W clisoroto pliiiit nuiso (^^). 

'I’lio cost intoj^ral oitn tilso bo written as the sum of A' into^rtils ns in the 
(lotoi’ininist i(! oaso. 4'lio control til time I/, must (»nty di'pcnd on tbo moasiirc- 
moiits inado until A,, tind the initial oslimaU' Hl,t)- 'I’l'is can bo ox])rcs.sod 
by rosti'idinu »(//,) to bo imaisurablo with icspoot to whore )\ is the 
mininial v-al^mltra ,m'i lent tod by tlio mo.asuromont soipioiioo (//(/,)./=(). I..,,. 

I) and tho initial ostiinato .f(/„) while the initiiil oslimato satisfies the 
ofpiation = 'I'lic usual <'hoioo of x(t^) as the moan of .r{/(,) 

satisfies this condition. 

llsitif' (7 fi), the oo.st functional (o) can l )0 put- int(j tho following form : 


J == /^r lr,y'Il.ry + }, V {x,,'Q/,j-/. -I- -h 

L 


U I 


4-v J 


-t- .f r('. iHQHDHt. H) di) 

h 


whore Q/,.. M/,. R,,. are given by 


f<\^'(i,h..)QAn<H'JHdt 

fl 


(On) 


(U) 



J <!)'(/. A, K,;^{/)n/./jd/ 




(9c) 


AV == f" {/.',(/) + r{L I, Min no, /,-)) >ii (o i) 

Note that. R/. will bo jto.sitivo definite since. positive definite. Similarly, 
Q/. will 1)0 po.sitivc .somidofinito. 

W'c shall now show that the lust two )orni.s in erpi. (!) r/) can be ignored as 
far as tbo minimisation of the cost functional is concerned : 


Lri))}wi ] 

Tho oontrol sofpionoo (a,, /,• (), t. .... ,V- Ij minimi/', os tbo (•c)^l finirtinmil 

•/ given by oqn. (!) n) if, and onlv if, it miuimir.o.s tbo following cost fimolioanl 


•/| -= A I Li-y //.i'y+ I ^ .r/ 'Qj,.!-/ y2.r,.' A! ), Jt s/^ Rxiij, 

i II 


(iOj 
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2! I 


Proof 

Krom (7 r), nulc tli!il f,) (|i‘[U'ih1.s only on (lie infivninnls of H’{.v) in llin 
interval (t;,, /|. vhich iii(> iml('|)t>iul('n( of and { U’(0. ^ 

«) is ^(/, //.) nf ( I find On the utliei' hand, ,r[l,J deinaids 

only on ‘"^d r{lo), ko that ^{1,1^) and ai-c inde]H>nd('nl 

whenever/^/;,.. 'I'lui.s, 


E.v,')\v'{t, i,)Q,mi, (,) dt = A’(.i-,')'y umnm>, h.)) = o 

u- u 


since has zero moan. Furthermore, the last term in (!) (i) i.s a con.stant 

i which (loos not depend on the ecmtrol socpienee ; henec, it can be e.vduded 
’ from the minimization. Ho the lemma follow.s. 

At thi.s .sta/re, in the case of the determini.sl ie rejiulalor, a preliminary 
feedback of the form H/.= - + would reduce tin; luadjlein into 

the standard reauiator problem. However, as .r^. i.s not ivailahle, the saim* 
cannot be dom; in tlii.s prubletn, ImsHnul, we now .show that minimization of 
the eo.st funetiona! ./j i.s ef|uivalent (o the minimization of the same expi'e.ssion 
wit]) ,T;i.’s replaced by their conditional e.xpcctution.s .f,.. 


Lemma 2 

'I’he control se(|U(>nee (»//*’, /■ = (), minimizes the eo.st fune- 

tiomif given by (In), if, and only if, it mi)iimlzr's the following cost func- 
tiomil T/., ; 




•V ■ I 


Uy'J/.v^ + ), -n.' <h.‘ I'lJh.) 


<•'-(1 


(11) 


whore .f^.= A’(.rJ )’d. 


Proof 

■Since is measurable with respect to usinu well linown theorems nn 
conditional e.vpectaiion.s, 

) ~ P a)) = ) /.)d/y_a;.) - ICx;'M j ii^. (1 2 ) 

Similarly, by lellitig .17 =s.r/ — W/. ; 

L(‘0, , QiJ',. h2x,,'Q^.r^ 

l.'sing the f.ict that /.>•,. ] 0, ue <ic| 

^'"(•'7. Qi‘'r) ■- ^'•'(•* 7 /( 1 ^/, *'7. /vf.fy 'Ch-r^j ) ; )) (1,'t) 

Since the (r-algelii-;is gemmated l>y .I'y and )'/ an' independeni, if lollow.s t hat 
(^’(•rV/.-r^j )V)-- A'(,rV^.rj.) a.e. (|.t) 
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Xolo tli.it llif> I iuht-hiuul .sido of (I I) doo-s not v.-iiy uilli i(j,. SnliKtilutinj' 
(12), (1 2) and (II) into (II), wo >i('t 




.V - I 






(- i/i(Xv7/.r-.v)+ ^ X (lii) 

k ^ (I 

.Situ'o tlu'pocomi torm in (In) doo.s not dopcnd on («^. A = 0, 1, A’— 1). it may 

bo droppctl as far as (lie minimixalion with rospoct to I, A'-l) 

is concorned. So reduces to ./j p;iven by (II)'. Ibis completes the proof of 
the lemma. 

'I’hiis, to find the optimal control for the origin. d eo.st funclionni •/, it i.s 
sufficient to find the control socpimiee which minimizes t|u‘ cost functional ./j 
of (II). Xoto that the eo.st ./„ deiiends oidy oil the eslimate.s of the st.-ite nfc 
the. sam])line in.stants. Ijookinit back at cfjii.s. (.S«), (.S/i) and (2). it i.s clear 
that tlie.se e.stimate.s will bo »iveii by the well-known Kalman filler eqiialinn.s 
(.]a/,winski M)7(), Kalman 12(10). We (‘an now introiluce a preliminary feedback 
of the form n,^ = — h'~' M + and embed the proldcm info the .standard 
(li.s(*rete linear (piadratic (lau.ssian ])mblem. Wc .sum up the.se resull.s in the 
next theorem. 


Theorem 

Consider the stochastic; 0|)(imjil control piadilmii deseribed in K 2. A 
unirpie control .socpienco {a,/*, L' — 0, \ X ~ 1) which minimizes the cost 
functional J of (4) exists and is given by 


»,*=~lAc-U//+(A.+ lVA-.,^Ib)->lVAb,pi),i.r, 


( 1 ( 1 ) 


where K; i.s the .solution to the Kiccati difference ecpiatiun 
with the boundary condition K^ =}[ 

where !’(/,, ,. /,). <Ty = <[)((„,, /j - and 

1=0. I, .... A'-l. 


Proof 

Infrodueing (he pi'eliminary feedback (ibiyne and .Silverman ID72, 
'l’homa,s.son am! Cook 1072) 
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the cost, fiinctioiia! hooomes 


21:3 


= A’ 


.V - I 




i- u 


(IS) 


'I’he o.>^t iinutc.s .r,. can l>c oht, lined l>y the Kalman fiilei- ei|iiations (dazwin.ski 
li)70, Kalman l!)(iO) 


■0,- 1 1 = " A + (' 1 , f'l ( I h) 

where is the white innovation .seiinenee. 

' Hence, the orii^inal constrained continmui.s prohlem i.s reduced to mini- 
mizing as given in (1.S) with tlie tlyi^amics of (I!)) where 71,. may depend on 
However, this is a discrete L(^(i pm!)lem and its solution is given Ijy (Hi) 
and (17) ; e.g. see Knshner (1971 ). 

Looking at eijiis. (Hi) and (17), it is seen that the optimal control law for 
this stochastic o|)timizatioii prohlem is the same as the (leterministic sampled- 
data regidatnr pi’oblem (Le\'is rl a!. 1971) witii ,r^. replaced hy its I'.^timate 
Therefore, the .septiration between estimation and control continues to liold 
for this constrained contiiuious stochastic ojAtimization jii'oblem. 


4. Conclusions 

In this |)api-r it is siiown that the sepai'ation princijile helueen estimation 
and cfuitrul continues to ludd for the stochastic saiiipled-data legulalor. The 
]irohlein is .solved by reducing the constrained eimlinuous .'tiH-haslic optiiid/.a- 
tion proldein into an unconstrained diserete stocha.s(ic one. 'I’lic results are 
(iljtaiiied iiy embedding the problem into the standaid discrete stochastic 
icgidator |)rohlem. 

Using similar tc(‘hiii(|ues to the oiu' describeil here, it can .also lie shown t hat 
the separation of estimation .and control continues to hold tor linear systiuns 
tising ?dh-oider holds and having eoinpidational delays in the control ioop. 
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Abstract 

In this paper, the geometric relationship be- 
tween the optimal solutions to the sampled-data and 
continuous linear quadratic regulator problems is 
investigated In a Hilbert space framework. It is 
shown that the optimal s.ampled-data solution, ex- 
cluding the response due to the Initial condition, 
1s the projection of the optimal continuous solu- 
tion onto the set of all solutions that satisfy 
the saaipled-data constraint. That Is, the optimal 
sampled-data solution is an optimal approximation 
to the continuous solution. In fact, it is shown 
that the sampled-data solution can be obtained by 
solving a sanpled-data tracking problem with the 
continuous solution as the desired trajectory. 


1. Introduction 

This paper Is concerned with the relationship 
between the optimal continuous and the optimal 
sampled-data Iinc.tr quadratic regulator problem 
(1], {2). Since the dynamical system and the inte- 
gral cost function used arc the same foi both the 
continuous and the sampled-data regulator. Intu- 
itively, the optimal samnlcd-data control must be 
an approxlm.tt Ion to the optimal continuous one In 
some sense. In this paper tlte precise relationship 
between these two optimal solutions will bo inves- 
tigated In a vector space setting. 

In the control literature, t!»e continuous and 
the s.impled-dat.a regulator problems have usually 
been treated using different methods: The continu- 

ous regul itor problem has been solved by u.slng 
Pcntryapln's mialmum principle, by using the 
H.anllton-Jacobi-P.ellman partial differential equ.a- 
tlon for the optimal cost function, atid by a few 
other methods. The sampled d.ita problem has been 
.solved bv convof'. Ing It Into an unconstrained dis- 
crete opt Irizat Ion problem througir the Integration 
of the cost I'uacllonal .uul t!io system differential 
equ.it Ions over e i.-h s.impilug interval and then ap- 
plvlug ilviiiml,' tir.rilni’, or the discrete mini- 

mum principle. i!i<* i el ,ii liuv.li I p between the two 
optimal sol .It l.ui:. is, thus, ob-icurcd due to these 
sepir.ale f.M nuil at I onr. , In ihl*; p.iper, the two 
problems are fot r u I .itotl In the samo fr.imowork by 
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using a Hilbert space approach. This geometric 
formulation reduces the problem to one of finding 
the element of minimum norm In two linear variet- 
ies In a Hilbert sp.ace. (This approach has been 
suggested for a simple control problem in f1}). It 
is shown that, excluding the unforced response of 
the system due to the Initial condition, the 
sampled-data control and the resulting state tra- 
jectory 1s the projection of the optimal contlnuoia 
control and its corresponding state trajectory; 
th.at Is, the optimal samplcd-ilata solution Is, In 
fart nn optimal approx Im jt Ion of the continuous 
solution with respect to an appropriate Hilbert 
.space norm. 


’ 2. A Minimum Norm Theorem In a Hilbert Space 

Before proceeding to formulate the problem in 
a Hilbert space setting, wc shall prove a general 
theorem concerning the elements of minimum norm 
of two linear varieties in a Hilbert space which 
will bo required in the following derivation. Wc 
now state the projection theorem and Its extension 
to Itne.ar varieties for case of reference; the 
proofs can be found In (3). 

l.c -ima 2. 1 (Rro lection Theorem) Let H be a 
Hilbert sp.ace .and M be a closed subspace of 11. Cor- 
respomllng to any vector x In H, there exists a 
unique vector m* In M such that m* Is the closest 
element In M to x in the sense of the Inner product 
norm. Furthemore , a necessary and sufficient con- 
dition that be this unique vector Is that x - 
m* bo orthogon.il to *1. (Ve will denote the projec- 
tion operator onto M by l.e., m* ■ P.^x). 

>'orol 1 a tv 2 . ? Let H and 'I be as In Lemma 2.L 
Lot X bo a fixed element In II and lot V be the 
linear v.arloty x + M. Then there exists a unique 
vector V* In V o? mlnlrnim notm. Furthermore, a 
necessary .and sufficient condition that v* be this 
unique vector Is that v* he orthogonal to the sub- 
space M. 

Tlif followlnr theorem d.'scrlbc*: the relation- 
ship h.'tw.-.-n the elements of minimum norm of two 
llni*ir varieties In a Hilbert sp.icc where the gen- 
erator siitif.pace of <uto v.irlety Is a subset of the 
other in^ f respond I ng •uihspacc , 

Theorem Let H bo ;i Hllhort spare .anil let 

M and ;i he closed suhep.nces of H such that N Is a 
sulisel of M. I.el x he a fixed element In II .and 
let V and U be the linear v.irlctles defined by 
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X + M and x + N ronpcrtlvrly. Then llicre cxlrtl i 
a°unlquo vuePor v* In V ;ind n unique vciior w* In 
U of minimum norm. Furthermore, w* c.m be obtained 
from V* by 

w* - x^ ♦ 0) 

where n* la the unique element of H defined by the 
relat Ion 

V* ■ x + m* (2) 

o 

Proof ! By Corollary 2.2, there exiRtp a u- 
nlquc element v* In V .ind a unique element w* In 
W of minimum norm. Since v* la In V, It must have 
a representation of the form v* ■ x + m* where m* 
la an element of M. Clearly, m* muSt be unique. 
Similarly, there exiata a unique element n* In N 
auch that w* ■ x + n*. Since N la a aub-Hllbcrt 
apace of M, there exiata a direct sum decomposition 
of M (4) given by 

M - N + N-L (3) 

where N la the orthogonal complement of N-^ with 
respect to H, and + denotes the direct sum. In 
order to prove that w* ■ x + P„m*, we have to 
show 

< x^ + n > - 0 for all n In N (4) 

where < . , . > denotes the Inner product. Since 
, < Pjji m*, n > la Identically zero, we have 

< x^ + " * " * o > 

Since m* has the direct sum decomposition 
■* • «« 

< x^ + n>“<x^+m*,n>“ 

< V*, n > ■ 0 (6) 

The last equality follows from Corollary 2.2 and 
the proof la complete. 

3. Formulation of the Probler 

In this section, we shall formulate the two 
regulator problems In a Hilbert space fr.smeworlc so 
that we can apoly the theorem of the last section 
to find the precise mathematical sense In which the 
optimal sanpled-data solution is an approximation 
of the continuous solution. Consider the linear 
dynamical system represented by the differential 
equation 

x(t) - A(t) x(t) + B(t) u(t) (3.1) 

a ^ t b 

y(t) - C(t) x(t) 

where x(t) Is the n-dlmenslon.nl state vector, u(t) 

Is the n-dlQonslon.il control vector, y(t) Is the 
r-dlmenslonal measurement vector, .niid x(;») Is the 
Initial condition. A(t), B(t), and C(t) arc matri- 
ces of appropriate order with continuous clement.s. 
The following cost functlon.nl will be used to .n- 
chlcve the desired system performance; 

J(x(t), u(t)) - (/‘’(y'(t)y(t)+ni'(t)R(t)u(t))dt)^^ 

(3.2) 

where P(t) Is a positive definite matrix with con- 
tinuous elements. We will denote the R*-v.ilued 


functions on la, hi thnt nro f.<iu;ir« Ini r.ib 1 c In 
the Leber.qiu' lioiiNC 1*v l”( i, !•) , 1.''., 

I.T(a,b) • (u(t): /*’ u'(t)u(l)<li ' ” ) (3.3) 

2 a 

Similarly, lot l.^(a, h) be i he pp.n<-e of H''-v,nlued 
square Intcgr.ibK' (um-tloif. on | ;i , b). I.’e will 
formul.ilo the twi» pioliltrs in I In’ Mllbi-rl •.p.irc^ll ■ 
Ln(n, b) X I." (a.b). III.- Carl •• .l.in produ. i of !•,(«. 
b) and L 2 (.n,b), wlih the Inm-i produrt defined by 

<(y^(t), uj(t)), (v^d). n^O))- ' fl (v;(t)y 2 (t)+ 
uJ(t)P(l)n 2 (t))di (3.4) 

so that the norm Induced hy the Inner product 3.4 
gives the desired cost functional o| 3.2. Since 
the solution of the differential equation 3.1 is 
given by 

x(t) - <(t,a) x(a) -f l(t ,s)B(s)u(s)ds (3.5) 
y(t) - C(t)x(t) 

where f(t,s) is the state tr.nnsltlon matrix of A(t), 
we will be concerned with the ordered pairs (y(t), 
u(t)) in II that satisfy the lntep,ral constraint 3.5 
for the continuous regulator. It Is easy to show 
that the subset II of H defined by 

M ■ {(y(t), u(t))cL 2 X l-jt y(t) « T(t,s)u(s)dsJ 

T(t,s) - C(t);(t,s)B(s) 

Is a subspacc, so that we can define the linear va- 
riety In It by 

V - (C(t);(t,a)x(a), 0) + M (3.7) 

Thus, the contlnucu) regulator problem Is one 
of finding the clcrcnL of minimum norm In V. To 
apply the Hilbert space theory of the previous, sec- 
tion, we have to show that H Is a closed subspace. 

To this end, let us define the linear transforma- 
tion L from 1 . 2 ( 3 , b) Into I. 2 (a,b) by 

L(u(t)) - T(t,s)u(s)ds (3.8) 

a 

Using Fublnl'r theorem. It can easily be shown 
that for all y In I.J(a,b) 

<Lu, y> - <u, T'(s,t) y(s)ds> (3.9) 

which proves that L Is a bounded and, therefore, 
continuous linear transformation. (See Theorem 5 
In (5), p. 48). We also note the linear transfor- 
mation L* from Lj defined by 

l.*y - T'( 8 ,t)y(s)ds (3.10) 

IS the adlolnt of L. So If (y ,11 ) is a sequence 
of elements In M converging to an element (y,u) In 
H, wc have 

llm (y ,u ) - 11m (/^ T(t,s)u (s)ds, u (s)) - 
n n n n a n n 

(/^ T(t,s)llm 11 (s)ds, llm u (s))rM 
a n n n n 

This follows since L Is a continuous operator end, 
therefore, M Is closed. 
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In the H.implrd-diCn prohlom, wo wtll ho plvon t 
p.irtltlnn (a ■ t|, . . . , t|,| - l>) of fa, b) 

siirh that over each sanplint; Iniorval tho control 
roinatnn conitCanC, l.e., u(t) ■ i t < 

‘k+r 

In this case, the subspace of Interest wtll be 
N ■ f(y,u) LjXh": y(t) ■ T(t ,s>n(s)ds, url.", 

u(t) - u(tk) for ^ t « tk^l) (3.11) 

Clearly, N lr> a subset of M, To sliow that N Is 
closed, wo note that the same linear operator L de- 
fined 3.8 Is used except now Its domain Is a subset 
of I,". So wc only have to show that this domain Is 
closed In L™* Clearly, tho set of all functions 
that are constant over each sampling Interval Is 
Isometrlcally Isomorphic to U'"xR'^ ... xR>n(Ntlmes) 
so that N Is closed. 

So the samplcd-data regulator problem Is one of 
finding the element of minimum norm In the linear 
variety W defined by 


Therefore, th>' optimal continuous solution Is given 
bv 

u*(t) --P’’(l) T’(s,t) y*(s)ds (4.5) 

or, equivalently 

u*(t) - -R‘^(t)B'(t) r (s,t)C'(s)C(s) x*(s)ds 

(4.6) 

It Is easy to show that there exists a posi- 
tive scmldcf Inlte matrix K(t) such that 

/J ‘.•(s,l)C'(s)C(s)x*(8)ds - K(t)x*(t) (4.7) 

for t In (a, b] 

where K(t) Is the unique positive semldeflnlte solu- 
tion of 

K(t) - -A’(t)K(t) - K(t)A(t) + K(t)B(t)R~^(t)B(d 
K(t) - C'(t)C(t) with K(b) - 0 


W- (C(t)*(t,a)x(a), 0) + N (3.12) 

Thus, we have cast the two regulator problems 
Into the vector space format of the previous scctlcn. 


4. Optimal Sampled-Dita Solution as an 
Approximation to the Optimal Contlntious Solution 

In this section, wc will first show that the 
Hilbert space formulations of tlie sampled-data and 
the continuous regulator problems In the previous 
section do Indeed give the standard results. Then, 
using the theorem concerning two linear varieties In 
Section 2., wc will state the precise sense In which 
the optimal s.amplcd-data solution Is an approxlnatlcn 
of the optimal continuous solution. We will also 
show that the optimal snmplcd-data solution can be 
obtained by taking the appropriate projection of the 
optimal continuous one. 

Consider the continuous regulator problem forru*- 
latlon described In the previous section by equations 
3. 1-3. 7. A necessary and sufficient condition for 
(y*, u*) to bo tlie element of nlnlmum norm In V by 
Corollary 2.2, Is that (y*, u*) be orthogonal to M, 
which Implies 


<(y*,u*), (y,u)> - 0 for all (v, u) In M (4.1) 

Since the Inner product on x L? Is the sum of In- 
ner products on Lj and L™ 

<y*(l). T(t ,r,)u(s)ds '+*u*(t) , R(t)u(t)»“0 

for all u In L"(a,b) (4.2) 

By using 3.'*, wc get 

T* (s . l >v*(.|)d:; , u(t)-'*- u*(t) , R(t)u(t)'»0 
for all u In I.J 
It follows that 

R(t)u*(t-) T'(sil) y*(s)ds - 0 (4.4) 


which Is the Rlccati equation of the linear contin- 
uous optimal control. 

Consider now the formulation of the sampled- 
data regulator problem described by equations 3.1 - 
3.5 and 3.11 - 3.12. A necessary and sufficient 
condition that (v**, u**) he this optimal solution 
Is by Corollary 2.2 that 


•'(y**,u**), (y,U)> ■ 0 for all (y,u) In N (4.9) 
which implies (after some manipulation) 

(/‘l (x’**(t) C’(t)C(t)r(t, t ) + u'**(t)R(t))dt 

‘o 

+ /^*J 

^t\ x’**(t) c’(t)c(t)dtr(t^,t^)) u^ + 

'•'t (x’**(t) r'(t)c(t)r(t, t.) u'**(t)R(t))dt 

t ^ 

+/|.^ x’**(t) c'(t)c(t)dtr(t2, 




+(/'n (x'**(t)c’(t)C(t)r(t,t„ ,) + u’**(t) 

R(t))dt}Ujj_j - 0 (4.10) 

for ail vectors (u^, u^ ..., ** 

where T(t, t.) ■ .‘(t ,s)B(s)ds. This Implies 

I *1 

that the terns in the brackets In (4.10) must be 
zero. Starting from the last term In 4.10 .and us- 
ing tho f.ict that r.**(t) ■ !(t, 
r(t, iN-l 1 can he shown that the optimal 

sampled-data control lav is given by 


whore r 


Qj - / 


“ / 


-(R^v 

r’ K r )“* 
1 i+i r 

r-'v >+*l')x** 

' 1 •'l+l 1 1 ' l 





(4. 

ID 

“ r(t 

i+r i'* 1 

l(tj^^pt^), and 



‘i+-i 






-^'(t.t^) Q(t) 

I(t ,t^)dt 

(4. 

,12) 

i+i 




S 

(t,tj)Q(t): 

(t.t^)dt 

(4, 

,13) 
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R - /'•♦^(R(t) -»■ r* (t (I (4.14) 

where K KallKricK llie «ll«cretp Klcc.it 1 equation 

16 ) * 




«1 ■ 


(4.15) 


1 - S - 1. 


0 with 


Si 


W* will mate the exact relationship between 
the two regulator solutions In the next theorrm. 

Theorem 4.1 Consider the continuous regulator 
problem described 3.1 - 3.5, 3.11, 3.12. l.et (v*, 
o'*) be the optiisAl solution to the continuous prob- 
lem given by 4.5 - 4.8 and let (y**, u**) be the 
optimal solution to the sampled-d.Tta problem given 
by 4.11 - 4.15. Then, excluding the response due 
to the Initial condition, the optimal s.rnplcd-data 
solution is the projection of the optimal continu- 
ous soltulon onto the set of ordered p.airs (y(t), 
u(t)) that satisfy the sairpled-data Integral con- 
straint of 3.11 or more precisely: 


(y**(t), u**(t)) ■ (C(t)t (t ,a)x(a) , 0) + 

+ P„(y*(t) - C(t) ! (t ,a)x(a) ,u*(t)) 

" (4.16) 


or, equivalently, (y**, u**) is the element of V? 
which minimizes 


/J[(y(t)-y*(t))'(y(t)-y*(t))+(u(t)-u*(t))’!l(t)(u(t)- 
u*(t)))dt (4.17) 


.'onong all (y,ui in v;. A r.ecessary and sufficient 
condition th.-z (>’**. u**) be the optin.il samplcd- 
data solution Is that 

/^t(y*(t)-y**(t))’y(t)+(u*(t)-u**(t))*R(t)ii(t))dt«0 
for all (y,u) In N (4.18) 

Proof : Equation 4.16 follows from Theorem 2.3 
with Xp • (C(t) ; (t ,a)x(.i) ,0) . m* - (y*,u*), and K 
defined by 3.11. 

To see 4.18, we note th.at be Lemma 2.1 
ll(y**(t) - C(t)J(t, a)x(a), u**(t))|| • 
minimum ij(y(t) - (y*(t) - C(t ) I (t ,a)x(a) ) , 
u(t) - u*(t))l| (y, u)iN (4.19) 

which Implies 

||(y**(t), ii**(t)) I 1^- minimum | | (y(t)-y*(t) ,u(t)- 
,,2 (y.“)‘W 

u*(t))|r (4.20) 

which verifies 4.17. Equailcn 4.18 follow:: from 
the last part of Lemma 2.1 and the proof is complolr. 

Theorem 4.1 impllc.s (r.io rqu.itlon 4.17) th.it If 
an optlmil s a-pl eri-dat.i tr.icV.lnr proMr*. Is .solve.! 
by any nethoJ(s>icli as dynamic proprinmlnr , illscrctc 
minimum nrlnclrle, etc.), where the der.Jn'H tra- 
jectory to be tracked Is the optimal conClnu<<us 


regulator solution, tlu-n tio- solul loo ol>l .i I ii. d 
would hi the H.IIII*' .IS tlie si'liilioii to llio oilr.iuil 
B.impled- d.it (I rer.ul.iloi piohli'i. 

5, Coin I II' I oil' 


The <..imn1 eil -d.il ,i and i ont I ii'i.'u'". llneii re|ii1.v 
tor prohlin ,iie f orw.u 1 i' i ■! In 1 ho s.in.o III lt» II 
space as m I iili .on norn prohlen.. The j-.i onu- 1 r i c ro- 
l.it tonslil |> hi tweon the two opllr.;l r.olutioiis 1'; In* 
vest i g.il od . It If -.liovn that tin- opt IimI s.i;q>Iod- 
d.it.i solution Is an nptlr..il .ipp i ox In .it iiui to I he 
com iniu>ii;. leciil.iLer solution in .m .tppi opii.ilc 
Hilbert space norm. .Specif leal I v , it is shown I lint, 
excludlnr the response due to the Initial condition, 
the optimal s.impled-dat.i soluljrn Is the projection 
of the npiininl continuous solution onto the set ol 
all solutions that satisfy the s.impi ed-d.it a con- 
straint. It should he also noted that extensions 
to the case whore (a,h) Is an Infinite Interval 
can be easily obtained with this method with slight 
modif icatluns. 
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Failure Accommodation in Digital Flight Control Systems by 

Bavesian Decision Theory 


Raymond C. Montgomery* and Alper K. Caglayant 
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\ design method for digital control systems whkh is optimally toleranl of failures in aircraft sensors is presen- 
ted. The function* of tbi* lysten are accomplished with software instead of the popular and costly technique of 
hardware duplicalion. The approach taken, based on M-ary hypothesis testing, results In a bank of Kalman 
filters operating in parallel. A moving window of the innovations of each Kalman filler drives a detector that 
decides the failure stale of the system. The detector caknlaies the likelihood ratio for each hypothesis correspon- 
ding to a specifk failure stale of the system. It also selecis the most likely stale estimate In the Sayesian sens* 
from the bank of Kalman fillers. The system can compensate for bardover a* well as increased noise-type 
failures by computing the likelihood ratios as generalized likelihood ratios. The design method b appikd to the 
design of a fault tolerant control system for a current configuration of the spare shuttle orbiler at Mach 5 and 
120,000 ft. The failure detection capabilities of the system are demonstrated using a real-time simulation of the 
system with noby sensors. 


Introduction 

T he most striking impact of new technology in aircraft 
flight control stems from the advent of the modern, high- 
speed, digital computer. Control concept.s previously con- 
sidered untractable can now be considered because of the 
flexibility and speed of information processing made 
available by this new technology. One important new poten- 
tial that exists is the ability of digital system to reorganize it- 
self to accommodate for failures in sensors and actuators. 
This reorganization is possible, provided there is enough 
duplication of function between the actuators or the sensors in 
a given control system. This paper presents a design method 
for digital flight control systems that will be optimally 
tolerant of sensor failures. 

Modern control methods allow one to determine the part of 
the state space of an aircraft that can be dynamically in- 
fluenced by a given actuator (the controllability subspace) and 
the part of the state space that a given sensor can produce in- 
formation about using state estimator theory (observability 
subspace). Reference I provides a good treatment of 
theoretical considerations involved in determination of these 
subspaces. Redundancy is provided in either sensors or ac- 
tuators when there is overlapping of the subsp.ices of the 
various sensors or actuators in a given system. For example, 
consider the longitudinal dynamics of an airplane. If there are 
three sensors on the aircraft, say an accelerometer, to measure 
normal acceleration, a pitch-rate gyro, and an elevator 
position transducer, and if the aircraft state is completely ob- 
servable from outputs of either sensor, then it is pt>ssiblc, 
using say a minimum order observer.* to csiimatc the 
behavior of one sensor based on ihe output of another one. 
Redundancy, in that situation, diH's exist and can be used by 
cross-checking state estimates obtained by one sensor with 
those obtained from ansMber one. 

Theoretical considerations for determining the absolute 
level of redundancy that exists in a given system were 
developed in Ref V Reference 3 also presented a failure 
detection filter designed to make use of the system redun- 
dancy. One limitation of that work was that no consideration 
Presented as Paper '4-21 at the AIAA 12th Aerospace Sciences 
Meeting, Washington, D.C., January .tO-February I, 1974; submitted 
April 12, 19'4; revision received March 21. 1975. 

Index category Navigation, Control, and Guidance Theory. 
•Aerospace Technologoi, Flight Dynamics and Control Division 
Member AI A.A 

tResearch \svocaie, Virginia Polytechnic Insiiiute and State 
li'itvcrviiv. in revidenxea: I aneley Research Center. 


of the practical noise environment of the sensors was made 
and failure detection depended on observing a steady-state 
bias in an error plane in a state space. For aircraft ap- 
plications, however, a design process is desired that enables 
rapid detection of failures during maneuvering transients and 
accounts for the normal operational noise environment of the 
aircraft and the control system actuators and sensors. 

A design method is presented for resolving both problems 
in that it accounts for noise in sensors and is capable of deter- 
mining hardover as well as increased noise-type failures 
during maneuvering transients. Incorporation of failure 
detection and recovery into an aircraft control system design 
is a joint detection, estimation, and control problem. The 
design method presented here produces a decision for delec- 
ting system failures which is optimal in the Bayesian sense. In 
addition, because of the theoretical development, one is able 
to account for uncertainty in the aircraft's stability 
derivatives, mass, inertia, and geometric characteristics. 
Although the method developed can be applied to both sensor 
and actuator failures, only sensor failure detection and 
recovery arc considered. 

The approach taken here uses ,)/-ary hypothesis testing with 
generalized likelihood ratios. The elements of this theory were 
originally developed at the close of World War II for a binary 
liyptMhesis testing problem of determining whether a radar 
return signal respresented a target or not. In that cave there 
are clearly two hypotheses — either there is a target or there is 
not. Theoretically, imc can assign a cost to cither failing to 
detect a real target or creating a false alami. ,\ pcrloiiii.iiice 
index can be constructed which expresses the cost ol making a 
decision based on a given radar return. This index can be 
ininimized by selection of threshold points ft>r deci'ii'n 
whether or lU't the retniii lepresents a tarci't. Fleiiieiits of tins 
probletn are outlined in Ref. 4. which also contains a brief 
description of the M-nry hypotheses testing and generalized 
likelihivod ratios. In this paper the vet of hypotheses used is, 
first, that all sensors are functioning properly and. then, 
M— ! further hypotheses stating that the rth sensor group has 
failed i = 1 ,2... ,\t - 1 . In the next section the theory for ap- 
plying .\.'-ary hypothesis testing to self-reorganizing systems is 
presented. Then, it is applied to an example aircraft problem. 

Sensor Failure Accommodalion rsing 
A/-ar> Ilypolhesis TeslinR 

Consider the equations ot motion of an aircraft to be 
represented bv 
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x^Ax + Bu + m/ (I) 

where x is an n-dimensional stale vector, u is an ni- 
dimensional control vector, w is a zero mean Gaussian white 
noise process with a covariance matrix W'6o(/-r). InLq.(l) 
matrices A and B are determined from the aircraft's s'.^bihty 
and control derivatives, its mass and inertia characteristics, 
and its geometric characteristics. The variable w may, but 
need not. represent turbulence. It may represent unceriaii.ty 
in the designer’s knowledge of the characteristics of the air- 
craft. Basically, it can be thought of as representing the error 
in calculation of x, given x and u. We will be concerned with 
the digital control of the plant where the control is con- 
strained to be constant with sampling interval T, that is 
u{t)mu ikT) for kT^K {k + I )T. By integrating the system 
differential Eq. (I) over each sampling interval,* we gel the 
discrete equations of motion for the aircraft 

x{k + n^*x(k)+ru{k) + w(k) (2) 

where M(/t)»M(kT), * = 

r= dsB 

.'o 

and w{k) is a zero mean, while Gaussian sequence with 
covariance E w(k) iJ)^Q ikj where 

0= [%’(s)H 4>(s) ds (3) 

.'0 

Let us assume that the control system has M-l sensor 
failure modes for each mode 

y{k) = C,x{k)-k-v,lk) i=l,2 kf-l 

w here v, [k) is a Gaussian while noise sequence where 
E\v,(k)] = {0.rh;. 
and 

The quantity rttj is an unknown (nonrandom) parameter 
vector. 

We shall solve the problem as if wi, were known and then 
use the maximum likelihood estimate of m, under the ith 
hypothesis. This procedure is known as generalized likelihood 
ratio approach in the communication literature.’ This ap- 
proach to failure modeling enables the designer to compensate 
for hardover failures of arbiiary magnitude. Increased sensor 
noise-type failures can be modeled by appropriate selection of 
the noise variances R,. 

For the normal unfailcd condition we will .isumc 
y(A) =Co-\ik) +c„(A) 

where £|i'o(A) 1 =0 and E|t'o(A )t'o Hence, for 

a system with three failure mivdes. as considered in the next 
section, we have four hypotheses to consider 

ffo- y ( A ) = CVv ( A ) + I’fl ( A ) 

v(A)=C,.v(A)-t-r,(A) 

//;;>(A) = C;.x(A) +r.,|A) 

M}.- y(k) ^C,x{k) +v,{k) 

where C, (/= I. 2. 3) is Co matrix wiih the rows corresponding 
to the /th group of sensors replaced by zeros. 


We will be concerned with the selection of the most 
probable hypothesis, based on a finite set of measurements, 
Y{K) = |y(/).v(2).>-(2)...>’(A) | . Todo this wc construct a 
Bayesian cost function for the .Af-ary problem 

Sf - / ,s# . ; 

i; v; Pn«(al«y) d« (4) 

iwl) J-O * 

subject to 

M / 

imO 

and where the sets Z„ i^O, are disjoint and their 

union represents the entire observation space. E/,, is the a 
priori probability of hypothesis H, being true, is the cost 
of selecting //, when Hj is true, and Pyj/y (al//, ) is the con- 
ditional probability density of the measurement sequence Y 
given that is true. The symbol i?, implies that the integral 
is carried over the decision region Z, in the observation space. 
Decision regions Z, are subsets of observation space such that 
if Y is in Z, then the hypothesis //, is to be selected. Note that 
the integral in Eq. (4) represents nothing more than the 
probability of making the incorrect decision of selecting 
hypothesis //, when Hj is true for i^j. So the Bayes risk («, 
represents the sum of probabilities corresponding to different 
decisions weighted by the a priori probabilities and the 
design weights C,j. The problem is to choose the luiundaries 
of decision regions Z, that will result in minimum Bayes risk. 
These boundaries are. in effect, switching hypersurfaces for 
the decision logic in the measurement space. 

The minimization of Bayes risk can be performed easily by 
rewriting the cost function (4) in the form 

2.^ I lA,(«) dor (5) 

ImO • 

where 

M f 

V^,(o)= P„,C„P,„,(orl//^) (6) 

/•o 

The Bayes risk is minimized by selecting H, at each point a 
in the observation ^pacc such that ^,(o) is the smallest of M 
possible values of i(,(o) (A = 0, Hence, the op- 

timal decision regions are 

Z,|a lil',(a) amin^i (a), OsksM- 1\ (7) 

From a computational point of view, it is convenient to in- 
troduce a dummy hypothesis H^f with a prion probability 
with .*'( A ) = r’o( A ) . Then, an equivalent 

decision criterion can be given in terms of likeliho<«d ratios. 
A,(o) 

.\,(i») =/’,!/, ("I//,)//’, i„(cil//,,) A/-/ (8) 

Dividing each 4,, in f-q. (6) by the probability density of 
YfKt under //„, we get an equivalent decision criterion in 
terms of the likclihrKvtl ratios 

Af I 

A,(a)= />„r„A,(i.) (9) 

n 

llien 

Z, = I a IX, (q) =minXj (o) . Os A s Af - / 1 (10) 

The advantage of using likelihood ratios is that the boun- 
daries of the decision regions are linear hvpcrplancs and not 
general hypiersurfaccs in the likelihood ratio space A/..\ t 


I 
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A*,.,. From the chain rule of probability densities and the 
Gaussian density of the observations, it can be shown’ that 
the likelihood ratio for the problem considered is given by 


A,mA)| 



(dei/?u)^'_ 

Idero,(Air 


evp 


K 



A >/ 




-y'{k)Ru 


V(A)1 




III) 


where r,(A) is the miiovaiioii* of the ineasiiremenis iiiuler the 
ith hy|H)ihesis given by 

r,{k) ^yik) -C,.i,(k\k- 1) -m,{k) (12) 

with (*!*-/)-£■ (jr(/lr)l>'(A-/). H,]. The matrix 
Q,{k) in Eq. (II) is given by 


Q,(k)^C,V^{k\k-nC:-kR, (13) 

where y,,(k\k-D is the prediction error variance of the 
estimate of the state x( A) under the ith hypothesis defined by 

F,,(AI*-/)=£| |jr(A)-jf,(AIA-7)l(ar(A) 

-3i,{k\k-l)\'\Y{k-l).H,\ 

In Eq. (12). the true value of m,{k) should be used to gel 
the exact likelihood ratio. Since this is not available, we will 
use the sample mean of 10, rh/(/), 0) ',>= /, 2,...k, that is the 
maximum likelihood estimate of nti at the Aih instant under 
the ith hypothesis. That makes A, a generalized likelihood 
ratio.’ 

To compute jf, (AIA-/) and F,,(AIA-/), M Kalman’ 
filters are required. A bank of Kalman filters opening in 
parallel has been used for parameter adaptive control in Ref. 
8. The filler equations are listed as follows for completeness 

jf,(A)=je,(AlA-/) + A',(A)r,(A) (14) 

x,{k\k-l)=*i,(k)+ru(k) (15) 

where jf, (A) is the estimate of the aircraft stale under the <th 
hypothesis defined by 



l-ia. I Fatill l>»lt-r«ni runlrol sysirm vlrwrlurt. 


Corsiderable simpliricalion occurs if one considers > 
I) Cg »0. RamiHcaiions of this assumption 
are discussed in the example to follow. Under those con- 
ditions the equations for A, may be modified without loss of 
generality to select the maximum of 

K tC 

E IO.(/)l-'/i E ^'t/)e^'(/)^0). 

/-/ 

1 = 0./,. ..Af-/ I 

where K is the total number of measurements used to make 
the decision. If the steady-state Kalman filter is used, we can 
select the largest of 

K * 

WnPn- - lnl<?,l-!A E r:U)Q:'r,U). 

* /-/ 

/*0./,...A/-/| 

Also, if the a priori probabilities of H, are equal, without loss 
of generality, we may take 


t,IF(A))= -lnig,l + V^ E 'lU)Qr'r,{J) (19) 

^ j‘i 


and select the hypothesis H, corresponding to the smallest r„ 
1 = 0,/ ... M- /. The next section demonstrates the application 
of this method to a practical problem. 

Application to Aircraft Flight Control 


.v,(A) =£(.v(A) ly'(A),//,) 

The filter gain A, (A) in Eq. (13) can be calculated recursively 
from the algorithm 

A, (A ) = X'tAk lA -l)C;Qr'{k) (16) 

where Q,(k) is given by Eq. (13) and the prediction error 
variance I',, (A lA - / ) is given by 

F,, ( A lA - / ) = -FF,, ( A - / ) ♦' -I- 0 (17) 

where F^, (A) is the filter error variance given by 

F, ( A ) = 1 / - A, ( A ) C. 1 F,, ( A lA - / ) (18) 

Note that, because of the special structure of the matrices 
C,, the unknown mean m, docs not enter the filter equations. 
That is, the estimates of the hypothesis conditioned filters will 
be exact. Thus, for each hypisthesis we have a Kalman filter, 
as previously inJicateJ, that can be used to determine the 
likelihood ratios, which can, in turn, be used to make the 
decision as to which hypothesis is most likely. The structure of 
the system is schematically’ indicated in Fig. I. 


The theory developed in the previous section has been ap- 
plied to the design of a control system for one space shuttle 
orbiter configuration at a Mach number of 5 and an altitude 
of 120,000 ft. Taking the state to be defined as ar= {p, 0, r, 
d) ' and the only effective control u = the aircraft equations 
of motion can be written as 


-0.0580 

0 
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TaMc 1 Ktalualion of H ( 0 0 0 0 


M' coitiponeni 

Level of 
certainly* 

F rror scale 

//,.>•« ! 0 

0 

1 

0 

O', (p component) 

2 

O.OS rad 'sec 

1 0 

0 

0 

1 J 

i*'] ( ^ component) 

0 

1 rad 





O’, (rcomponeni) 

2 

0.01 rad' sec 

r • 

0 

0 

0 

M'^((i component) 

3 

0.(X)l rad/sec 

1 




- - - - 


■ 

//..» = ; 0 

0 

0 

0 


* 0 iniplin atHoluic ccruiniy, I implies a high level of ceriainiy, 2 im- 
plies only moderate certainty, and 3 implies not too sure. 


The selecfion of the variance for the last equation in- 
volves consideration of I) the uncertainty that we. as 
designers, feel related to our knowledge of the equations of 
motion, 2) the relative scales of the variables, and 3) the en- 
vironment, with regard to turbulence, under which the vehicle 
must operate. We will only consider the first two items here. 
Table I shows the authors’ interpretation of the level of cer- 
tainty and scale considerations. Concerning the level of cer- 
tainty, it was felt that the 0 equation was well understood 
since it represents a well-known kinematic relationship. A 
high level of certainty was assigned to the 0 equation. On the 
other hand, the^ and t equations were felt to be better defined 
than the equation. Turning to scale considerations we have, 
in effect, equated an error of l/O.OS in the computation of p 
to one of l/O.OOl in the computation of 4- The W matrix 
selected is constructed from the elements of Table I as follows 

W'=diag (2(0.05)^ 0(1)*. 2((l.01)*. 3(0.001)*) 

The discretized equations of motion using a zcro-ordcr-hold 
with a sampling interval of 0. 1 sec is 


H,:y. 


10 0 0 
0 0 10 
0 0 0 0 


jr + Uj 


Variances of the measurement error are taken, consistent w ith 
current technological capability, to be 

/fo = diag((0.05)*. (0.01)*. (0 01)*) 

Failure covariances are assumed to be larger than the unfailed 
ones. (The behavior of the resulting system will, however, be 
illustrated for both statistical failures— increased varian- 
ce — and for hardover failures. This capability is a direct result 
*'f not assuming a zero-mean measurement error in the failure 
slates.) 7 he values of/?,. /?,. and/?, used are 

/?, =diag (0.025, 0.0001. 0.0001) 

/?, = diag (0.0025, 0.001. 0.0001) 

/?, = diag (0.0025. 0.0001. .01) 


■ 0.9798 

-0.0002 

0.0267 

-0.5752 


’ 0.2240 ' 

0.0992 

1 

0.0587 

0.0310 


i 0.0139 






x{k)+ 1 

0.0021 

0 

1.002 

- .0740 


0.005.36 

0.0497 

0.0006 

-0.0862 

0.9887 

. 

0.00538 


and the discrete variance matrix for the process n(A ) is 


0.4757 

0.04757 

-0.0066 

0.0236 

0.04757 

0.00654 

0.00100 

0.00309 

-0.006 

0.00100 

0.02015 

-0.00185 

0.0236 

0.00309 

-0.00185 

0.00211 


x(IO) ’ 


which was evaluated using Eq. (.3). Note that, because of ihc 
sampling, even though the « equation was considered ab- 
solutely certain, uncertainty does result in the 0 equation of 
the discrete model. Also, the components of the plant noise 
vector arc correlated in the discrete model. 

For illustration, consider that the vehicle has three sensors: 
a roll-rate gyro, a yaw-rate gyro, and a sideslip indicator. 
There will, therefore, be four hypotheses to consider, as 
follows 


For each hypothesis the state is observable but, given the 
measurement errors and uncertainties in the vehicle equations 
ot motion, each hypothesis has a different capability of 
estimating the state of the aircraft. Fiencc. embedded in the 
theory is the consideration of the capability of any given sen- 
sor group, corresponding to each hypothesis, to estimate the 
state of the aireraft. This is reflected in the error covariance 
matrix elements of each hv|H>thcsis. As an example. /; | (/) 
-/>) •’ 1 under each hv|>oilicsis is indisatcd as the (1 , I) element 
ol the error covariance matiix and is 0.00075, 0.0015, 
0.(H>()S2. 0O00R7 for //„. //,. //,. and //,. respectively. As 
ex|vcied, //fl has the smallest value of F| (/>—/>)■’ 1, in- 
dicating that this hvpothesis, if true, can prcnluce the best 
estimate of .Also iiuhcaied, however, is the fact that //, 
produces the worst estimate Again, this is expected since //, 
corresponds to deletion of roll 1 ate gyro information. 

In this example, 7he ll.ivesi.m lisk weights C,, are taken as 
(■„ = I foi / and (’„ I .Also, steady -state Kalman filters 
arc used so that Lq. (IK) is applicable, l or the example here, 
Lq. (18) becomes (using a memoiy si/eof five samples) 
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During control system operation the scalars r, should be using 
the innovations r, of the Kalman filter bank stored over the 
past live samples. Then, the hvpothesis corresponding to the 
minimum r, should be selected. 

The behavior of the system has been studied using a hybrid 
computer facility in which the equations of motion of the 
vehicle were programmed on an analog computer and the con- 
trol system was mechanized in a digital computer. Figure 2 


illustrates the unaugmented step response of the vehicle to an 
aileron input. This aircraft is a nonmi''imum phase system in- 
dicated by roll reversal. Also, the aircraft possesses a large 
coupling of the Dutch roll into the aileron response. Digital 
feedback was employed at a cycle lime of 0. 1 sec using feed- 
back gains (-4.9, 0.4, 14.5, -6) for (p, 0. r. 0), respectively, 
to the aileron. The gains were selected to be constrained to a 
control system operating with only roll control. Figure 3 
shows the response ot the closed-loop aircraft to the same 
pilot step input wlicn slate variable feedback (perfect 
mcasTiremenl of each state) is employed. Considerable im- 
provement in flying qualities could be obtained if yaw control 
were available. Figure 4 illiistruies the same step respoiue 
using noisy measurcinenis and accepting Up. No actuators 
and sensors have been failed in Fig. 4. In Fig. 5 the responses 
of the system arc indicated for ilte case where Hj is true but 
for each hypothesis being accepted at different times. The 
failure mo^ considered in Fig. 5 is an increase in 
measurement noise. Note that at the start of the record Hp is 
selected and produces poor characteristics, as can be seen by 
comparing the Up true portion of the roll-rate trace of Fig. 5 
with that of Fig. 4. Had there been no failure, those traces 
would be almost identical. When //, is selected at ap- 
proximately 5 sec, poor characteristics are still produced. 
However, when Hj is selected at approximately 10 sec the 
system moves to a normal operation, only to return to its poor 
characteristics when Hj is selected at approximately 15 sec. 
This figure illustrates the effect of accepting hypothesis H, 
when H, is true. It indicates the effect of cost selection of the 
terms in the Bayesian risk function. Figure 6 shows the 
fault tolerant system in operation when failures of increased 
noise type are introduced. By looking at the {p, r, 0) 
measurements, it can be seen that the following sensor failure 
modes have been simulated; {Hp, H,, Hj, H)\. The plot 
showing the hypothesis accepted indicates the performance of 
the detector logic. Figure 7 deals with the detection of har- 
dover failures. A hardover failure in the beta sensor has been 
simulated. Note that, although detection logic is able to detect 
sensor failures in all cases quite rapidly, the detector logic 
takes a longer time to reject a failure hypothesis when the 


H||. 2 Rrvpofl«« of unaug- 
mcnird aircrafl to aileron 
virp. 




Tiim.i. mc 



Fig. 3 Response of air- 
craft to step pilot input 
when digital stale variable 
feedback (with no mea- 
surement noise) is used. 
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Fig. 4 Krsponie of rlo«cd-loop lytlcm with nols> mr»iarrmrnts ufi' 
d«r normal operalion. 






TlW.I M( 


Hf . 5 Rcsp<*nsr of riosrd-loop »\lrm drmnnxiralini; rffrcis of w- 
rrpilng h)p<ilhrMs Hg. Hj, Hj, »hrn Hj i« true. 


system is already in one. Further, note also that only the 
steady-state Kalman filters are used and overall performance 
may be improved using time-varying Kalman fillers. 




X 
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fig. 6 Operation of fault l.derani s)ftcin during failures resulting in 
increased noise. 




!.•» t tK 

fig. 7 Operation under saturation hariloirr lailure. 


Conclusions 

A digital fault tolerant control system dcsipn that ac- 
commodates for aircraft sensor failures has been presented. 


Fach sensor failure mode and the normal operation of sensors 
are modeled as M different hypotheses. Then, using the 
Ifayrsian Af-ary hypothesis lesiinc approach. .i deieenon logic 
is developed that results in .i h.inV nl \f K.Tlm.in lilt »s The 
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decision logic, which uses M generalized liKelihood ratios, 
selecies the hypothesis that minimizes the cost of making a 
wrong decision in the Bcyesian sense. The likelihood ratios are 
calculated from a moving window of the innovations in each 
of the Kalman filters. The istimaie of the state corresponding 
to the hypothesis selected by the detection logic is used in the 
control system. Tiie design system is capable of identifying in- 
creased noise type and liarilovcr-type sensor failures. These 
capahihiK's are demonsiraled using a real-nine hybrid 
siniulaiion lor a space shuttle vehicle lateral dynamics. 
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Abstract 

This paper is concerned with the parameter 
adaptive least squares estimation of random proces- 
ses. The main result is a general representation 
theorem for the conditional expectation of a random 
variable on a product probability space. Using this 
theorem along with the general likelihood ratio 
expression, the least squares estimate of the pro- 
cess is found in terms of the parameter conditioned 
estimates. The stochastic differential for the £ 
posteriori probability and the stochastic differen- 
tial equation for the a posteriori density arc 
found by using simple stochastic calculus on the 
representations obtained. The results are special- 
ized to the case when the parameter has a discrete 
distribution. The results can be used to construct 
an Implenentable recursive estimator for certain 
types of nonlinear filtering problems. This is 
illustrated by some simple examples. 


1. Introduction 

This paper is concerned with the parameter 
acaptive cstic.ation cf random processes cerrupted 
b) additive r.cise. The problem may be briefly 
stated as fellows. From a given (possibly uncount- 
able) collection of random processes with known 
distributions, one process is observeo with adJiiive 
nsise. The a priori prebabliity, that a specific 
random process in i!iis collection is observed, is 
spccit.cu f >r c.'tvii one in the collecticr,. The 
prr'c.i” .« to fir.J the 'tasi ‘queres csiirate of ^hc 
olser ’d rrocess. 

IhcT-: i» n Ijrre cl^tr of p’.y'j'.crl rre’-lc-s 
tret c »r hf :rns <Je:ei .r. thi« rcn.nxt. For 
irstancc. :..rt detection and cstiir;iticn [9], [la], 
astir.it-cn .r..-.r unccrtc.i.t/ [I'-j, jo.i.t t.-tir tJen 
..nc identification [10], peremetor rdaptlvo self- 
organizing c.-r.trj’. [15] . [16], [17]. 

Partr.ctwr aJip.ive cotir.nion of tlscretc 
Causuiar. piuccr.nes with lii.-ar dynamic mcdels liaa 
been tre.itej by Hap.lll |12]. l.alniotis [10] lias 
Invistle.ifi'il till’ p.irnmeti r ndapi Ive esilmotlon 
problem for cuntlnuouJ C.?ur«irr. procssrscs with 
linear c/i... .Ic iii-.d-’l!.. Ctl.c relalcw wcik ulong 
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these lines can be found in [18] - [21]. The first 
systematic treatment of Joint detection-estimation 
in a general setting was done by Middleton and 
Esposito (9). 

In the mathematical theory of probability, 
related works can be outlined as follows. Parameter 
adaptive estimation for a specific Jump Markov 
process with a finite number ntates has been stud- 
ied by Uonhan (22). The same problem for a two 
dimensional Markov process with tlic nonobservablc 
component a jurp Markov process and the observable 
component, a diffusion process, has been resolved 
by Shiryaev [23]. Related work on the estimation 
of arbitrary stochastic systems can be found in the 
work by Kalllanpur and Striebei [5] and for Markov 
processes in [24]. 

This paper generalizes the results of [10], 
[9]. The approach taken is along the lines of 
[13], [5]. Tl'.c problem is fcrmulated in the usual 
Bayesian decision theoretic framework. In the 
Second secticn, a mathematical statement of the 
prcVlim is given. In section 3, a general repre- 
sentation tbeoren is proved for the ronditional 
cxpc-ctatlcn of a random variable on a product prob- 
ability space using the properties of Radon- 
Mii.odyn derivatives. This theorem also yields a 
RaJon-I.'ir'.odym represcnlallon for the a p' stc r iori 
prob.ih il ity which can be tb.ougbt of as a pcncral- 
i/.ed c .yes tbecrer... By using the reprcf nation 
thiorems alone, -tth int general lik.ciihc'.->d ratio 
c,':;- rc; -lion cf K;i i lath-Dunc.an [6], [7], the para- 
meter idaptive estimation problem of random pro- 
ci Is solved in section 4, Ihe rain thoorc- 

oi ti..3 sccilcn cm be- tb.ougat ot as a gcnciolin 1 
partition tl-.oerem. In section 5, by using simple 
stcc:..stlc calculus, the- stochastic differential 
cc.rc ponding to the a pester ' ert probability -td 
till st3ci...stit di f f«, rcrtial c3u.Uion for tie a 
pof t' r iot'i density arc found. In section 6, the 
r< arc specialized to the case when the p-ri* 

m. ter pare Is cri.nt..bly infinite, l.c., wli’n l‘t 
par.-iiiii'ti. r lu.s a cl sc re it- dtniribut ton. ibe tesuU* 
of ti.. ■ secllci. iiuludi’ till- c.a;.e of finite patu 
a. < 1 .: p.'cc such ;if M-.iry liypotbescs. So lion 7 

in<;‘.'. k " so-e i 1 1 '.s t rat I on .inJ ..ppllcat ietis. 

1. I..I c -«’nt of the Preble.*- 

vill b, ..nCiMud with tw.' -jf li : 1 ■ :ic' .- 
a! lit’' -paces (21: (. j , Pjl .ird 

ill I .yi'i i- I -• :i’ 1 . > will eerie p';. . : e t • 

,-..r tor e..j will di i ’n..io ’ -f ' 

pat.. Ki..*t; t-» ct .1) .and will be L.;.tto‘- 
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«nd Hi rc’K|iri'l Ivrly. ^X|>(■c I .ii I on wllli ro';|>ocl lo 
munsiirrs Pj ;iml 1’^ wllj bo don'>lod by l'| /nul i.^, 

Wf Khali asMiini- ihai tln-r*'' cxls(:< a voctor Wlcni r 
proci'SK v(l , *i) on (Oy , Aj , P.' ) wlih zero m an and 
Increment III eov.irlanco I.; dv(i) Jv'(s) ■ R'(t) dl. 
Kor each Pi.'i, wo will be given two veclor random 
procoKBCK TpU. n) and VpCit f^uth that ihi’ future 
Ir.crementh ot the Wlcner'^prccess v(t, 'i) will be 
independent of t lie past of v(t, n) and *^^(1, n) wltli 


• ^O 'e 


(8,n)ds + v(t,n) 


tclO.Tj (2.1) 


Vic will assume tliat for a fixed t t (0,T1, 
yg(l, n) and 2g(ii h) arc Jointly measurable In 
(6, n) on the product prob.'ibility space (SI] « 02 • 

A] X A 2 r Pi X P 2 ) 60 that we can now define the ran- 
dom processes y(t, n. 6) and z(t, n, C) on 
(fJi X (J2t ^1 X A 2 , Pi X P 2 ) by 

y(t,6,n) ■ yg(t,n) t 10, Tj 

*(t,e,n) - *g(t,n) ectJi . ( 2 . 2 ) 

So that the observation model becomes 

y(t,e,n) - /* z(s,e,n)ds + v(t,n) tt(0,T]. (2.3) 

The problem is to find the conditional expec- 
tation of s(t,6,n) given a sample function of 
y(t,0,n) in terns of the conditional expectation of 
Z|^(t,n) conditioned on the sample function of 
yo(i|l)> fhc implicit assumption in par.-imcler 
adaptive estimation is that the parameter condi- 
tioned estimates are easily computable. 

Given two arbitrary sets ili and fl2 • shall 

denote the product of fJi and fl2 * ^2 1 vhich 

is, by definition, the set of all pairs ( 011 , 102 ) 
where the points ui and 012 run through flj and il2 • 
respectively. If Y is an arbitrary mapping of 
(i) X r.j into an arbitrary space, we shall denote 
Y,. , the section of Y at u] , to be the mapping 
) 

defined on R 2 by Y (uj) “ Y(o.i, -;). If A is an 

‘ w; 

i-ibitrary set of Hi * fi, we denote A , the section 

‘ wj 

C'f A at L.J, to be the subset of Ci; defined by 

A “ (u2 c (>2 ■ (wi , U2) c a}. 

“1 

Nov the problem can be slated as follows: Find 

the conditional expectation of z(t, n , 0) given a 
sample function of y(t, »i , 6) in terms of the condi- 
tional c'lpectation of the section of z at 0 condi- 
tioned on the sa.Tple function of the sect! .1 of y 
at P. 

V^e shall use the following definition cf 
conditional expectation [3]. Let g be a random 
variable on the probability space (r?, A, P) and Y 
be a measurable transformation fiom (.'?, A, p) into 
a measurable space (V,6). If exists, then 
L(glY ■ y) is defined to be t’.ie nc .isurable mapping 
from (V, G) into the real line, R, sucli th.it 

/ g(w) dP(u.) - / i:(gl Y«y) dlY"'(y) 

Y" (B) B 

for every UcB 

whore Y \b) ~ (u>; Y(ui) t B) and PY '(•) is the 
meas..te on R defined by PY"^(D) ■ P(Y"*(b)) iinU G 
is the Borcl sets of R. 

In the paranu'ler adaptive •-•Mimation we will 
be concerned with two dilfereiu conditional ixpcc- 
tatlons. If y is tlic set of all P.'* valued contin- 
uous functions on [0,T] and G is tlii Poi.l k. is on 
V', then we can define tl>c measurable tiapplpp, Y 
from (tlj X E; . » A 2 , P 1 X Pp) into (V', G) such 


Y : (fJ, X {I 2 , Ai X A;, Pi X P 2 ) - (V'. I>) 

by Y(0, n) ■ (y(t, n, 0 ), 0 ^ i i T) . 

Similarly, we can define the ne.i.'in-bli luipp j ng 
where 0 is some fixed point in the pai .imelti space 

ni 

Yg : («2. A 2 . P2) - (V. B) 
by 

Yg(n) - (yg(i , n) , 0 t T) . 

Thnt is, Yg is the section of Y at 0. Now in 
this setting we can talk about 

*(t,y) •• E(z(t, e, n) 1 Y(e, n) - (y(s), 0 ^ s ^T)) 
and, for a fixed 6 c Oj , 

Zg(t, y) - E 2 (Zg(t, n) 1 Yp(n) - {y(s), 0 £ s ^ T)) 
which will be defined correspondingly (P - Pj x Pj) 

/ z(t, e, n) dP(e, n) - / z(t, >•) UPY"’(y) 

Y"*(D) B , „ „ 

for every B t B 

/ z(e, n) dP 2 (n) ■ / y) dP 2 Yo"'(y)- 

Yg-l(B) B 

The problem is to find z(t, y) In terms z^(t, y) 
where 6 runs through the parareter space (!]. 

3. Characterization of Conditional Expectation 
on the Product Prcbablllty Space 

In this section, we will prove a general 
theorem coniernir.g paru'i'.tr adaptive estimation. 
Namely, a represeniaticn will be found fur tlic con- 
ditional expectation cf a random arlable on a 
product probability space using t!ie properties of 
Radon-Mkodym derivatives. This theorem also gives 
a representation for the a p orter i eri probability. 
Tlie results will be repeatedly used in the rest of 
the paper, llic n prcscntai ion tlicorems arc gener- 
alized Bayes tlicorcrs. 

Tlieorsm 3.2 I.et (.], A;, Pj) and (^2 1 A;, P 2 ) be 
two prob.-tbill ty spaces and (V, t>) be a r.casui able 
space vlicrc 6 is gtneroted by a count.iblc class of 
sets. Denote the product proi^abi llty space 
(til X fl;. Ai X A 2 . Pj X P 2 ) by C(, a’, P). Let Y be 
a measurable transf oirai ion from (E, A, P) into 
(V, 8) and g be a random variable cn (G, A, H) 
whose expectation exists. (g is quasi-lutegrable 
in Keveu's sense [?)). Let tig denote the measure 
induced on (V, B) by the section of Y at 0 c Ej, 

1 . e . 

■ •* 2 (Y- ”\b)) for every B t 5 


If tlierc exists a c-flnlte measure on 
(Y, 8) .inJ .1 set Nj in Aj of Pj-reasute zero such 
that pg <‘ Po for 0 I Ej - Nj, then the conditional 
expectation of g given V is 

E(g(0, n)/Y(0, n) ■ >) - 

/jj b: 7 (gp(n)lYp(r.) - y)--'- (> ) dl’,U>) 

M . 
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UWGLVAL non IS 
f POOR OVALITY 


where !'(•) 





In the RaJoii-Nlkoilym 


derivative ol wq w.r.t. K denotes expectation 
w.r.t. the meai.ure I’ (l.e. Pj * P 2 )« f'd (uj denotes 
almoni everywhere with respect to tl»e Pleasure u> 
Proof ; See Appendix. 

C orol larv 1.3 Under the asr.unptlons of Theorem 3.2, 
Tor Pj - .ilmesi all 0 e fli and << p, the R-N 
derivative is given by 


dp 


(y) 


dp 


(y) 


0 


(m) 


So that 


n.v 


E(g(0, n)|Y(e, n) - y) - E 2 (gg(n)|Vg 

diT <y) 


y) 


(3.4) 


Also , 

- 

P(A, * fl 2 |Y - y) - ^ (y) dPi(O). (3.5) 

Proof ; See Appendix. 

Note that p(Aj * ^ 2 !^ “ y) ?. p osteriori 

probability of the event Aj in the p.iramctcr space 
conditioned on Y. P(Aj * 112 !^ " y) defines a 
measure « oa Aj-sets for each y c V by: 


»(Ai , y) • (^1 “ O2 1 ^ ■ y) 


(3.6) 


Uc also note that the rcprcsent-ition for the 
_a posteriori prob.ablllty is a general leed Bayes 
theorem. To sec tills, let ()} be countable, 

£J| " { 61 , 02 ,... , 0 .... ), and fi 2 • R and let p 
be Lebesque measure on R, tlicn by 3.3 and 3.8 


"(0j, y) 




o 

I r.COjIjA;,) 

i*i o 


(3.7) 




Note tliat j — ^ is by definition, the conditional 

a;. 

o 

prcb.ibility dcii.sity of Y given so tliat (3.7) is 
one form ef t'.’.c celebrated theorem of Bayes. 

We shall now apply tlie results of the Theorem 
3.2 and Corollary 3.3, to the p.rranetcr adaptive 
csLlniatlcn of random processes. 


4. Par.imotor Cond Illum'd 
r.sLlrMl ion of Random Processes 


In til'.*! rci ti.in we will apply tlie results of 
tlio previous si'ciion to p.ii .t et’r id.ipllve ostlma- 
tion of r.ii'.dc*' pi o :es:if}i . Ilie 1 1 'pi'eseiital Ion 
tlii'oiom tor Llu’ I .'I'.l i t J on.'il expi .:l.il Km of llio I;l;:t 
section will tie n:i d along wiili the general llkell- 
iiood r.itlo 1 /.pres'iioii of K.ill.atli and L'nnc.in (6), 
(7), 120], t-‘7|. l.'e now state the general llkcli- 
liood ratio re;, II Its in llio next Jeiumi vliich Is a 
sjip.hljy ditfereiii 1 »■•it.u emenl i’! Tlieorem 2 In (6). 

A'.L (-li I'l. i’l) '‘"d (d_i . A... l'^) be two 

p roll I'li Illy .pire... Asminie th.it fi'r each 0 c fli, 
i.o .arc giviu Ihii e me i.;urable vei ier r.indo.-i proces- 
ses (taking tlicli v.ilnes In R") -■.o(t, n) , yi)(l, n) , 


.ind v(l , m) wltii r, ( i> and t t ( 0 , T] sucli tliat 
v(t, n> is a in 10 me.ni Wiener process with incre- 
ment. 1 1 cov. 111. nice 

K2 dv(i) dv'(s) - P.(i) Jt 

and tiie iliree random processes arc related by the 
eqn.ilien 

Vjj(t, n) - * 0 ^"* 'i) ds + v (t, n) . (4.1) 

I.ol Yf be itic space of all R'^-valucd contlnous 
fuiiclioits on ( 0 , t] and be the Borcl sets of 
Define ilic nioa.ur.iblc t ranafurmat Ions Y^g and Y^y 
from (fl^. A;, P^) Into ()^^» K^) by: 

: (« 2 . A 2 , P 2 ) - (Yj, B^) 


Y^g(n) ■ (yo^*» 0 i • i 

•• (fl 2 . A 2 . P 2 ) - (V^. 8 ^) 

Ytv(n) ■ (v(s, n), 0 8 £ t) . (4.3) 


Let 
diiccd on 

P,Y and PjYty-* be 

(\. B^) by: 

the measures 

In- 


(D) - P2(Yj_,j‘‘(B)) 

B c 

(4.4) 


(B) - P2(Yj/'(D)) 

B c B^ . 

(4.5) 

If. for 

some 0 c fli, 



^7 11 

*0(t. n) 1 1 1 dt < - 


(4.6) 


and 


* 9 ’(t| h) dt < « (4.7) 

and the future Increments {v(t, ri) - v(s, n)) are 
Ind. pe:. dent of tlie c-alpeora generated by (v(t, n), 
Za(i. '!)• 0 X ^ s), then P.iYjo~'<< P 2 Y^^~* and 
the R-N derivative is given by 

t 

(y) - exp{/ z ' (s,y) R-l(s) dy(s) - 

dP^Y^^-' ° e 

1 \ ds}lP2Y^^->] 

(4.8) 

where 

Zq(s,v) • l.^(Zg(s, r.) I Y^jj • y(C), 0 < C ^ s) . 

(4.9) 

Proof : See Tlieorcn 2 in [0] and also Tlicorcm 1 In 

Tv). 

Tlicorcm 4 . 2 Lot tlie assumptions be as In Lemma 
4.1. .\lso a.ssuaic that n) Is a measurable 

function in (t, n, 0 ) so tli. 1 t we can now define the 
me.i-airable random proi-e.i‘:ei. z(i, n. 0 ) .md y(t, n, 
0) on tlic pioducL preh.iliil i Ly e.picc (l). A, P) ■ 

(W I • l.K , A I ' .\ > , 1*1 X 1 * 2 ) liy 


: (t, 0, 

n) ’ z (i 

. ' 1 ) ) 




w 

[ t r (0. 

T), 0 t 


y(t, 0. 

n) ■ >-y(t 

. M)J 


Tlie tlirei- r.i.idom 

procivif;i“5 arr 

related 

by 


y(t, C, n) » 7.(s, », n) d:. *• v(l, n) 


(4.10) 
n c 1 I 2 • 

(4.11) 


t I 10, T]. 

14.12) 


'lO 


Let be the nc.iMiralilc tr<ni'‘forni.il li'ii 
: («, A. P) -* (Y^. B^) 

Yj^(e, n) •* (>•(». P. n), 0 ^ s ^ t). (A. 13) 

Note thfit Is the section Y^ nt P. If thcie 
exists a set Hj in Aj of Pj measure r.iro r.utii that 
for all 6 t Rj - Nj, the aKSumplii-'ns in Lemma 4 arc 
satisfied, and t(z(t, n. 6)) exists, Ltien 


E(t(t, e, n) 1 Y^(o, n) - y) - 
*g(t. y) Ajj(t, y) dPi(O) 

/jjj \(t. y) ‘*‘* 1 ^ 0 ) ‘ t J 


(4.14) 


where^ *0^^* y^ ^y 

y) - E 2 (*j,(t, n) I Yj_j, - y(0, 0 < C 1 t) 

(4.15) 


and the R-N derivative A„(t, y) is given by 

t * , 

A.(t, y) - (y) - exp (/ z_’(s, y) R"'(s) 

8 dPjY^^-* ° 

dy(s) - ■j/q *e’(*» y) R”*(s) *p(s> y) • 

(4.16) 


Pro of : by Corollary 4.3, vv have 

Vllii 

dP Yj> A^(t, y) dP,(fJ 

By Remark 4 in the proof of Iheorem 3.2 
dP Y 

/ A (t, y) dP,(0) (y). 

The result no’J follows from l.emm.i 4.1. 

Comments : The R-N derivative A 0 (t, y) is the like- 

lihood ratio for the detection problem (with 0 
f Ixcd) 

hg! yQ(t. n) • * 0 (s, n) ds + v(t, n) 

h^: y 0 (t, n) ■ v(t, n) . 

A(t, y) in Corollary 4.4 is the L.R. for the compo- 
site hypotheses testing problem 

h ; y(t, n, 0 ) ■ *(s, n, 0 ) ds + v(t, n) 

h„: y(t, n, 0 ) - v(t, n) . 
o 

Note that 


Proof ; For all 6 c fij - Nj by Lemma 4.1, we have 
P 2 Yjq”* << ** 2 ^tv~** derivative is given 

by Ap(t, y). We can now apply Theorem 3.2 with 
y - y,. B - 8,, Po(-) - P2Yt0-‘(*). W(-) - 
P Yt-*(-), vA') - P 2 Ytv‘M*l. 

Corollary 4.3 Under the assumptions of Theorem 4.2, 
for Pi-alnost all 0 c tlj , P2yt0~* ** ^ ''t~^' 
the R-N derivative is given by 

dP2Y 


tO 


-1 


dP Y^“^ 

So that 


(y)‘- 


Ag(t, y) 


/fjj A^(t, y) dPi(C) 


E(*| Y^ - y) - ie(t.y) 




-1 


dPY 


-1 


Ipy^-M 


(y) 


(4.17) 


Also 

P(Ai 


dPi(e)lrY^-i) . 

fijl " y) “ 

dPi(0) (PY^-M . 


Ai jpY^-1 


(y) 


(4.18) 


(4.19) 


Proof ; Follows from Corollary 3.3. 

Corollary 4.4 Under the assumptions of Theorem 4.2, 
we have the following representation for the K-N 

derivative ■■ (y) 


dPY 


-1 




-1 


A^ft, y) 


dPY^“l 

A (t, y) 

where 

dP Y 

A(t, y) - 

(y) 


dPjY 


* tv 


i'(8, y) R“'( 


(4.20) 


f‘ z( 
o 


A^(t, y) dPi(0 

represents a normalized likLlihood ratio. Tliis 
normalized LK is the ratio of tlie conditional 
probability density ol the p.^rameter conditioned 
on titc obsetvation and the probability density of 
ti»e parameter; that is, 7g(t, y) is the normalized 
conditional density conditioned on the observation. 

Mote also tliat PfAj * il 2 I ^t “.Y) Ibc _a 
posteriori probability of the event Aj in the pata- 
meter space conditioned on the observation. Since 


P(Aj » ri; I Y, 


dP,Y 


-1 


y) - /X 


to 


dP Y^ 


(y) dFi(O) 


we liave proved tlie existence of the conditional 
probability density of the parameter. 

Theoremn 3..: and 4.2 unify a nutber of knevn 
results in tlie literature. For intianct, in the 
case, where the parar.etcr space contains two 
points, each one corresponding to the presence or 
the absence of a signal, we obtained the relation- 
ship betW' cn the optimal c-st' latc of tlie signal 
under uncertainty and the optimal estimate of sure 
signal in the least squares sense derived in ref- 
erence [9). In the case, where the signal and the 
mcacurcncnt noise arc Indi pendent and the signal 
is a CauKsl.an signal with a linear dynun.ic modi 1 
we get the M'ulti: In |10). 


The ixp 1 e!.;. i 'll derived ler ihe conditional 
cxpfclallun in this serilon is useful in the case 
of fixed observation time Interval. If the data 
is coming cuntinously, a stochastic differential 
equation Implementation is more practical. In the 
next section, by lining simple Ito calculus on the 
representations obtaiin-J in this section, we 
shall find the stochastic dl I fercntial e.ju.atlons 
that the .a postirinrl probability and probability 
density uatisly. 


(4.21) 


A1 


J. Stocliastlc Differential I'quatlcna 
for the A Posteriori Probability 


The results of the previous section ore useful 
when tiu* observation Interval Is fixed. Since In 
most technoloi;l cnl appllcallons tlie data la obtained 
continuously, the differential form of the results 
is more practical to Implement. In this section, 
we shall find the stochastic differential equations 
for the a posteriori probability density and a 
stochastic differential representation for the a 
pos tcrlorl probability by a simple applicatloii of 
Ito*s differentiation rule on the normalized con- 
ditional probability density representation in 
Corollary 4.4. 

Theorem 5.1 If the assumptions of Corollary 4.4 
hold, then normalized conditional probability 
density 


dP Y^“* 


(y) 


^g(t, y) 


exists Pi-almost everywhere and la the unique solu- 
tion of the following stochastic differential 
equation 

dX (t,y) • X-(t,y)(z (t,y) - z(t ,y)) ’R"> (t) (dy (t) - 

i(t,y)dt) (5.1) 

with ths initial condition unity. 

Proof ; From Corollary 4.4. 

y) ■ 

exp{ y) R"’(s) dy(s) - 

exp{ z’ (s, y) R"*(s) dy(s) - 


\ y) R"*(s) y) ds} 

. (5.2) 

J /' z' (s, y) R~‘(s) z (s, y) ds) 

Let R bo tl.e identity matrix with no loss of gener- 
ality. 

Since the required partlals exist and are con- 
tinuous, by Ito'a dif ferentation rule [23] we get 

dX^(t, y) - Xg(t, y){Zg'(t, y) dy - z(t, y) dy 

- J (c.y)*g(t,y)dt + (t,y)r(t,y)dt) 

* y^g('ty)(*g(t,y) - z(t,y))' (Zg(t,y) 

- r(t,y)) dt 

” V-(t,y)(Jg(t,y) - i(t,y))' 

(dy(L) - S(t,y)dt). (5.3) 

To show uniqueness, let Xj and Xj be two solutions 
of 5.3, then 

X^dXj — XjdXj XjdX2dX2 — X2dXjdX2 

d(X|/X2) ■ / + j • 

X 2 X 2 

Dy 5.3 it is clear that X 2 dXj - XjdX 2 “ 0. By 
using 5.3 and stoch.nstlc calculus 

XjdX 2 dXj - XjX;(r^(t,y)-z(t,y))’(Zg(t,y)-z(t,y))dt 


X 2 dXjdX 2 - X 2 X,(z (t,y)-z(t,y))'(zg(t,y)-r(t,y))dt. 


Ue, tlicrefore, have 


, ».<o 

X2(0) 

• XTW 


Xi(0) 

*2(0) 


0 for all tcIO.T). 

(5.4) 

for all t in (0,T]. 

(5.5) 


So if Xj(0) ■ X2(0), we must have Xj(t) ■ X 2 (t) 
for all t in (0, TJ. Detailed arguments can be 
found in [13]. 

From the stochastic differential equation chat 
the normalized conditional probability density 
satisfies in Theorem 5.1, it is now easy Co find a 
stochastic differential representation for the » 
posteriori probability in P(Aj * Oj I Y • y). 

From Equation 3.6, we know that wfAj, y) - 
P(A] X D 2 I Y " y) defines a measure on A| sets for 
PY”i almost everywhere, and w << Pj. The R-N 
derivative Is given by 


ds 

dPi 


dP Y^"* 


(y) 


(5.6) 


By using Theorem 5.1 and equation 5.6, we 
shall find the stochastic differential for Che a 
p osteriori probability. 

Tiicorem 5.2 Let the assu-ptlons o( Corollary 4.3 
hold. For a fixed parameter set Aj , the £ 
posteriori probability 

«^(Ai X 02 I Y - y), 

admits the following differential 

dwt(Ai, y) - (zg(t.y) - dSj(9,y) 

R-l(t)(dy(t) - j(t.y)dt) (5.7) 

with i>o^Ai* y) ■ PiCaj) is the same 

<*’’t(Ai. y> " f'^'A! dt^(e, y) - "t(A,. y> 

£(t, y)j R-J(t)(dy(t) - J(t, y) dt). (5.8) 

Proof ; See appendix. 

In the next section, we shall speclallzo ths 
results to the case where the parameter space is 
countably infinlts. 


6. Discrete Parameter Case 


The real advantage of parameter conditioned 
approach to estimation occurs when the parameter 
space is finite or countably infinite aince the 
solution may then bo rc.^'llly implemented on a digi- 
tal conputcr. Tills will be tlic case when the p;ua- 
rncier lias a discrete distribution or it has been 
suitably qu.intlzod to bo put on a digital computet! 
The par.nmeter adaptive npproacli will of course be 
more rewarding when the p.jramcter conditioned csCl- 
matos can be easily obtiilni'J. Vc now give the re- 
sults for the discrete p.irancter case. 

Tlieorem 6.1 

Lot the asrumptlon of Ttioorcn 4.2 hold. If 
the purnmetor space is countnbly infinite (or 
finite), let 

Dt ”{0j, 62 •••• 8 > 

* n 

Chen 


. . . } , 



E(*|Yj-y) 


1 


«l 


i. Ao.ii.y; 

- ^ [rv ,*| (6.1) 

): i’i(o.) Ao.(t.y) *■ 

1-1 * i 

where 

JejCt.y) ■ ,i|ytOi - y(«). o i • i o (6.2) 

and the R-N derivative 1> given by 

‘*’’ 2 ^ 7 ^ 

Ag (t.y) ■ — — (y) ■ e*p {/* («,y)p”* (»)dy(s) 

‘ dPjY;!, i 

, - 7 *1 (e,y)R"'(8) *g (B,y)ds). (6.3) 

A O 

Proof : Follows from Theorem A. 2 by Integrating 

with respect of Pj over Rj. 

Corollary 6.2 Under the aasumptlon of Theorem 6.1, 

for all e. c Rj, PjVjg « P Y"^ •«»» the R-N deri- 
vative Is^glven by J 

— — 1 — 
dPY 

So that 


ill Ae^(t.y) P,(o^) ' 


Ipy’*]. (6.4) 


dPoY'i 

2 te. 


E(z!y -y) - r I (t.y) (y)P, (e ) [PY"^ 

* ^ ^ ®1 dPY * ' ‘ 

' (6.5) 

P(6, X f.,|Y -y) - Pi(0.) ^ (y) (FY /] . (6.6) 

j ‘ t J dry"' ^ 


Tlieoren 6 . 3 

I'r.der the assumptions cf Theorem 5.1 and 
Theorem 6.1 the a p ostcrl oi l probability for a fix- 
ed value of tlie pararcler a^nitu the lollovliig 
stochastic differential fer 

rj(?^.y) - P(e^ X RjlY^-y) . (6.7) 

(dy(t' - z(t,y)ct) (6.6) 

with - P; (Pi). 

Freof ; Fellows form Tncorts 5.2 by carr^'ias out 
the ir.tegritlcn with respect to Pj . 

In practice, since z(t,v) will be computed by 

*(t,y) -Jj ’’t^'l'y^ • 


It is tl.trefcre of interest to Itok at the exis- 
tence £_nd unlojcncffi oi t.-ie sclutlon cf t..c 
nlte set cf t ir-jli ore e*as erj.-.tlcr.! that I^.ctio;. 
fc.£ inpllts. 

Trcorc". 

t.-.cwr tr.t as>u?tlcn cf Ti.tcrir. 6.1, t!.c set 
of ^ r otteric rl prubebllltlce 

•> f-:,.yi, 1 ■ 1, 2, 3, ...) 

\ A 

Is t .« •.-r.it'-iC stl'.tic.n cf t.'.e Iriii.ltc ct.rt sys- 
tem cf stctl.estlc differential c';uf.tlc^.^ 


*J*j(h^,y) - " J >y ) tj > “j.'-i “o 

y)) R-'(l) (dyd) - /g (t.y) »^(0j.y)dl) 


1 - 1 , 2 ,... 

with ■ rj(Uj). 


Proo f : See appendix. 

Tlicoietri 6.1 .iiid ii^pJy .in liuple nenl able 
estlm.itor structure which cunslnts cf two p.arts; 
(1) a nonadaptlvc part In whlth par.ineter condi- 
tioned cstlnatcs arc found, (2) r.n .idaptlvc p.irt 
in whleli the a p osterio ri ptoh;’'llltlro ore found 
If the differential rquatlun'i for the p.-iromotcr 
conditioned estlratcd arc avall.ihlc, tltcn these 
estinates along with the (llffercntl.nl equations 
for tlie a posteriori probahl 11 ties (Hieoicn 6.4) 
provide a recursive solution to the problem, in 
the next section, we sliall give some examples to 
Illustrate this estimator structure. 


7. Examples 


The usefulness of par.nmctcr ndcptlvc appro.nch 
to estimation prublems h.is been Illustrated In 
several papers [10], [12], [16], [20], [21], [22]. 
In this section, vc sh.nll outline some further 
possible appllc.ntlons of paro.mctcr conditioned 
approacli to estir.atlon. 

Example 1 Consider tl-.e nonlinear stochaEtlc dyna- 
mic system described by the following stoch.istl-: 
differential equations 

dx ■ x t-^ dt + dw 

do - 0 


with the observation node! 
dy ■ x 0^ di > dv 

where x(0) Is a Gaussian random varl.Vole vith mean 
u and variance and f (0) If an ari'itr.iry randc- 
var;l.ible with a dl.scretc dlsttlL ..tlon f. (a^). Tne 
Wiener procesres and v and l.jvc rcro rear, and 
u.nit variance. Also, x(C', t (C), v(t), tr.d v(t) 
are indcpendcr.t. Ly .nppl;. ir.p liiecrtr? 6.1 ai.i 6.**, 
with 6(0; as the jeranettr ve get the least squarei 
estimate for x(t) 

x(r) -^1^ r^(t) x^(t) 

where x.(t) Is the solution cf the f’alcar.-I j:y 
filter 


^ x^(t) - Oj^^Xj^(t) + Kj^(l) (y(t) - o^’x^(t)) 


where Vj^ft) Is the tclutlcn of F.iccatl 
with xi(0) - ii and i'^ft; • Oj'Vj 


ot 


Vj(t) 


with \’i(o) 
bill tit.-. 


Of^V^C.) - - ! 


.ind t ic sat of r 


cquatlr-n 


1 orf probt 


(r,(t), 1 ■ 1, ... • 

Is the ur.lT.it solution of the icilcvlng sysit- of 
ftochastlc differential equ.itltr.f 
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<Jn^(t) - »^(t) x^(t) Oj’wj(t) Xj(t)) 

(dy(t) ajJ »j(t) ij(t)dt) 

with (0) » • 

Example 2 In this cxanplo, we shall conaldor the 
rondon CclcRraph signal z(t), with values -1 and 
transition density, o Juirp/second where a Is a dis- 
crete random variable with a range of values (aj, 
®2» ••• and a priori probability distribution 

(PjCoj), J • 1, 2, M). The observation model 

Is taken to be 

dy(t) ■ z(t)dt + dv 


‘*“0 

f(o. y) “ j— (y) e c Oj - Nj . 

“o 

Proo f ; Soo 14) pp. 616-617. 

Proof of Tl i corcn 3.2 ; Let us define the signed 
rcaburcs v and (vq, Otflj) on (V, B) by 

v(B) - / g(e, n) dP(9, n) Bc8 (A-1) 

Y"‘ (B) 

and 

^ dP^Cn) BcB (A-2) 


where v Is a unit Wiener process. By applying 
Theorem 6.1 and 6.4 we get 
. M 

z(t) - E w. (t) z. (t) 

1-1 

where the parameter conditioned estimated z.(t) 
satisfied the stochastic differential equation de- 
rived by Wonham in [22]: 

dz^(t) “ 2 Zj^(t) + (1 - z^2 (t)) (dy (t) - z^(t)dt) 


which in turn derives the stochastic differential 
equations for a posteriori probabilities given by 
Theorem 6.4 

dx.(t) - x.(t) (z . (t) - E x,(t) z (t)) (dy(t) - 
1 11 J J 

j-1 J 

with 

n^(0) - Pj (Sj^). 


where Yo<n) are the sections of Y and g at OcOj . 

Wo have from the deflnltlona of conditional expec- 
tatlona 


E(g(e, n)|Y(0, n) - y) - ^ (y) 

E2(g/n)|Y^(n) • y) - (y) . 

8 


(A- 3) 
(A-4) 


So we are to prove that 

‘*''0 ‘*‘'0 

8 0 

dv , , 

— d^7~ (a-5) 


P.cmark 1 

''(B) - v^(B) dP,(A) . (A- 6 ) 


8. Conclusions 

The problem of optimal parameter adaptive esti- 
mation for random processes is formulated In the 
Bayesian framework. A general R-N derivative repre- 
sentation for the least squares estimate of a ran- 
dom variable on a product space Is derived. The 
representation theorem Is applied to tlie optimal 
parameter adaptive estimation problem for random 
processes to find the least squares estimate of the 
observed signal and the ja posteriori probability of 
parameter conditioned estimates. Tlie stochastic 
differential equation for the a posteriori proba- 
bility Is derived. Tlie results arc specialized to 
the case where the parameter has a discrete distri- 
bution. The .ipproach is illustrated by simple ex- 
amples . 


A ppendix 

Tlia .-ippendix cont.ilnn the proofs that arc not 
^tven In the text. To prove nicorcm 3.2, wc shall 
need the following Icoima. 

U-m 1 3. 1 Let (flj , Aj , Pj) bo a probability space 
oi;l ti) bo a mi ;>sur.iblu space wltcrc li Is gene- 
rated by a count.iblc clojs of sots. Suppost that 
for each 0 in Hi we arc given a measure Ug(y) on 
(.', u) such that, for fixed BcB, Uq(B) Is mca- 
suc.V.'Ic In 0. If there exists a sot in A^ of 
7i Tvasuro zero sucli that, for all 0 in Slj ” Nj , 

..I « Uq where ug » measure on (V, B) , then 
chore Is an Ai x 3 mensurable function f >uch that 


This follo-vs from Fublnl's theorem and the fact 
that Y 9 -‘(B) - (Y-l(B))a so that 

v(B) - ^V^B) dP;('')J dP,(0). (A-7) 

9 


Remark 2 E .(g.jfn) |Yo(n) ■ y) can be chosen to be 
Aj x B - measurable.* 

Since V 9 << ug and i..-. << jq. then by the chain 
rule of R-N derivatives (Theorem 32. A In (IJ) 








(A-8) 


ft 

Since ug and Ug are measures, Is non- 
du ‘‘“o 

negative. Since (yc/: — (y) - 0) has p measure 

di(g ”^0 ® 

zero and (yeV: -j-— (y) ■ •) has u and consequently, 
o ® 

Ug -measure zero. It follows that 


0 


% 

du 

o 


< 


K). 


(A-9) 


So 

du„ du f du ‘*^0 ' (A-10) 

dvg du 

By Lemma 1.1 and — can be chosen to be 
o 0 
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dVg 

A| X 6 • neanurublo, ho that can lio cliobcn tc 


‘*“0 

be a Aj X B neasurablc function* 
Rem ark 3 

dv 


du 


^ (y) - /„ E 2 ( 8 ^(n)|Yj,(n) - y) ^ (y) dP,(n). 

® ‘ ° (A-11) 

From Remark 1 we have 

w(B) - BgCn) dP 2 (n)] dPj(o). (A-12) 

Let ge(n) - 8 o(n) - gy(n) where gt «"«* “^e the 

poaitlve and negative parte of gn(ll. then 


(A- 13) 


v(B) - /jj t/y-ljB) Re 

^16 

-/(j IV‘(B) ‘«*’2<n)) dPi(o) 

From Che definition of conditional expectations 

‘ (A- 14) 

From the chain rule of R-N derivatives (Tlieoren 

du. 


32. B In (D), 

du^(y)) dPj(e) 


du (A-15) 

"^n 'V 2 (R“el^ 6 -y> dT 

• o 

By Lenaa 1.1 and Remark 2, the terns E 2 ^Rd > 


du 


can be chosen to be A, x B - 


measurable. By applying Fublnl's theorem for non- 


negative functions. 


du, 


VC can write that v(B) 


A o 


6v 


(A-16) 




Since the difference Is well-defined, at least one 
of the Integrals should bo finite and, consequently, 
the integrand corresponding to that Integral must 
be finite valued almost cvcr>~.'hure Pj x Ug. So the 
following tern Is well-defined. 


E2(EelVe-y) ^(y) - E2(g;hVy) (y) 


du. 


- tE 2 (EeIVg-y) - E 2 (g;|Y„-y)J (y) 


e' e 


du 


Fron Theorem 6*5*2 in [3] this expression 
■ ^zCgJifg-y) (y). 

o 


(A-17) 


(A- 18) 


So that ^ 

® (A- 19) 

since the term In brackets is B-ncasurablc by 
Fublnl's theoren and since the R-N derivative is 
unique a.c., 

dv 


du 


dT " V ^2^Rel^9‘y'^ dT • <^-20) 

O 1 o 


Rt rewrk 4 

’ -^n, dir 

O 1 o 


This follows from Remark 3 liv Hfliinr. r (’ ^ 

X Hj (*'.'>) where l;ij x l*'c ch.iiMcterlstlc 

functloit of tlie set W, x 1.’, Tills can be sciii as 
follows 

v(B) - X r - P(Y"‘(C)) 

- U(B) . (A-22) 

Kwr since v << u and v << u^ (Remark 4), we have 


dv du _ dv . . 

du du du **o' 

o o 

Since 

« > IT ‘ - W 

o 

VC have 


dv 

du 


dv y du 

du^' du 

0 o 


[u] 


(A-23) 


(A- 24) 


(A-25) 


The result now folloi’S from P>emark 3 and Rcrark 4. 
Proof of Corollary 3.3 ; Let h^Cy) denote the right 
hand side of 3.3. Fron Ttieuren 3.2 we have 

E(g|Y-y) - E 2 (gg|Yg-y) hp(y) dP,(C). (A-26) 

With g - Ij^^ where Aj c Aj 


This proves the last assertion in the statement of 
the corollary. 

Integrating both sides w.r.t. PY~ * 

^B X dr,(»)d/y). 

(A-28) 

Applying the definition of the conditional expec- 
tation to tlic L.li.E. and Fublnl's theoren to the 
right h.ind side (h,/y) Is Aj x 8 - r.easurabl- ami 
non-negative) , 
we get 

V'(B) n X c, 

Using Fublnl's theoren on the left hand side fElvrs 
'a, ■ -^A dPi(O). 

(A- 30) 

So for Pj-alnost ever>'i;hcre on f?j we have 

.^15> - FjfY^'o)) - /g hp(y) di.(y) (u). (,\-31) 

Since h^(y) Is S - measurable for Pj-alnost all P, 
It must be that 

‘‘“e 

dT 

Proof rf Thcor c r 5.2 ; There is no l. ss of ymera- 
llty In asbuninr, that R Is the Idvntltv matrix. W* 

shall first .show that the following equality holds 
alno.U suitly . » t .» 
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By applyli.g the differential rule on the rc- 
preient.-itlon of Aq In I'heorem A. 2, we have 

Ag(t,y) - Ay (O.y) - /‘ Ay<B,y) ig<».y) dy(«).(A-34) 

Integrating with respect to Pj over flj we get 

A(t,y) - A(0,y) - Ay(s,y) *y(*,y) dy(s)l 

dP,(9). (A- 35) 

On’ the other hand, by an application of the 
differential rule on the representation of A(t,y) 

In Corollary 4.4 we get 

A(t,y) - A(0,y) ■ A(s,y) i'(s,y) dy(s). (A. 36) 

Also, by Theorem 4.2, we have 

/{, *9<“ty) Ay(s,y) dPj (e) 

«(.,y) - 

So that the right hand sides of A-36 and A-3S must 
be equal almost surely which proves A-33. 

Now by integrating Equation 5.1 in Theorem 5.1 
over Aj with respect to we get 

/- X»(t,y) dP,(e) - /; \ (O.y) dP.(9) (A-38) 

^Aj^^o " *(*.y)l' l<iy(*) - *(«.y) 

ds]} dp,(e). 

with r} 5 place of z(s,0,n) in 

A IS ,yl 

A. 33, we get the following equality. 

I C *0^«.y) ‘IPl 0 

" I/- * 9 (».y)dPj( 0 ) ] dy(s). 

So that 


Proof o f Theor em ».4t The existence and uniqueness 
ot tlilM type ol iiiflitlie order clochastlc differen- 
tial equation han been studied by Rozovakll and 
Shlryaev (28]. To prove Theorem 6.4, we nhall apply 
Theorem 1 In [28]. To this end, we have to show 
that »t^**l» y) flass M. It la clear that, 

for each 1, »t^®l»y) continuous with probability 
one. This follows from the continuity property of 
the Ito Integral which states that the points of 
continuity of the random process defined by tho 
Ito Integral are the points of continuity of the 
Wiener process with resnect to which the Integral 
Is defined (4). tloasiirahlllty conditions are 
satisfied due to the definition of y)* 


I »,,(9..y) • 1 

l-l ' ^ 

Also 

T - - 

P { / ( £ r (s,y) X (s,y) ]2 ds < • ) 
o 1-1 1 1 

T . 

- P ( / z*(s,y) da < • ) - 1 . 

Therefore,. xt(9i, y) Is in class H. By 
Theorem 1 in [28], the system of equations 6.9 has 
a unique solution In the sense if t and t are two 
solutions with ’'jj(9^.y) - *Qt®j»y)t i 
then 

P { sup sup I »^(0j^,y) - (6^.y) I > 0 ) - 0. 

1 t 
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OPTIMAL PARAMETER ADAPTIVE ESTIMATION OF STOCHASTIC PROCESSES 

by 

Alper K. Caglayan 
(ABSTRACT) 

This study Is concerned with the simultaneous detection and least 
squares estimation of vector random processes. The problem is formu- 
lated in the following context: A random process, out of a countably 

infinite collection of (not necessarily Gaussian) vector random 
processes with known distributions, is observed with additive white 
Gaussian noise. The a priori probability, that a specific random 
process will be observed, is specified for each one in the collection. 
The least squares estimate of the random process that is being observed 
is to he found in terms of the hypothesis conditioned estimates. 

It is shown that the best estimate is the linear combination of 
the hypothesis conditioned estimates weighted by the a posteriori 
probabilities of the hypotheses conditioned on the observations. A 
Radon-Ni kodym derivative representation is derived for the a posteriori 
probability by using the specific structure of the product probability 
measure for this problem. It is shown that this Radon-Ni kodym 
derivative can be expressed in terms of the Radon-Nikodym derivatives 
of measures induced by the random processes in the collection with 
respect to Wiener measure. By using the recent results on likelihood 
functions, an expression for the a p osteriori probability is found in 
terms of the conditioned estimates. In this connection, an extended 
version of the partition theorem of parameter adaptive estimation 
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is proved. The unique stochastic differential equation, that each 
a pos teriori probability satisfies with its associated a priori 
probability as the initial condition, is derived for the case of 
finitely many hypotheses along with an expression for the conditional 
error covariance in* terms of the hypothesis conditioned error 
covariances . 

The results are applied to the parameter adaptive estimation 
problem in linear continuous and discrete stochastic dynamic systems. 

In the continuous case, the solution is also obtained through an 
alternate approach using nonlinear filtering theory. An application of 
the theory to the design of a digital flight control system which is 
tolerant of sensor failures is presented with real-time hybrid computer 
simulation results. A review of random processes and statistical 
decision theory is also included. 


